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Abstract 

Complex  systems  typically  possess  a  hierarchical  structure,  characterized  by  contmuous- 
variable  dynamics  at  the  lowest  level  and  logical  decision-making  at  the  highest.  Virtu¬ 
ally  all  control  systems  today  perform  computer-coded  checks  and  issue  logical  as  well  as 
continuous-variable  control  commands.  Such  are  “hybrid”  systems. 

Traditionally,  the  hybrid  nature  of  these  systems  is  suppressed  by  converting  them  into 
either  purely  discrete  or  continuous  entities.  Motivated  by  real-world  problems,  we  introduce 
“hybrid  systems”  as  interacting  collections  of  dynamical  systems,  evolving  on  continuous- 
variable  state  spaces,  and  subject  to  continuous  controls  and  discrete  phenomena. 

We  identify  the  discrete  phenomena  that  arise  in  hybrid  systenis  and  review  previously 
proposed  models.  We  propose  a  hybrid  control  model,  coupUng  differential  equations  and 
automata,  that  encompasses  them.  Our  unified  model  is  natural  for  posing  and  solving 
hybrid  analysis  and  control  problems. 

We  discuss  topological  issues  that  arise  in  hybrid  systems  analysis.  Then  we  compare 
the  computational  capabilities  of  analog,  digital,  and  hybrid  machines  by  proposing  intu¬ 
itive  notions  of  analog  machines  simulating  digital  ones.  We  show  that  simple  continuous 
systems  possess  the  power  of  universal  computation.  Hybrid  systems  have  further  simu  a- 
tion  capabilities.  For  instance,  we  settle  the  famous  asynchronous  arbiter  problem  m  both 
continuous  and  hybrid  settings.  Further,  we  develop  analysis  tools  for  limit  cycle  existence, 
perturbation  robustness,  and  stability.  We  analyze  a  hybrid  control  system,  typically  used 
in  aircraft,  that  logically  switches  between  two  conventional  controllers.  Stability  of  such 
systems  has  previously  only  been  tested  using  extensive  simulation;  we  prove  global  ^ymp- 
totic  stability  for  a  realistic  set  of  cases.  Our  tools  demonstrate  robustness  of  this  stabihty 

with  respect  to  “continuation”  of  the.  logical  function.  „  . 

We  systematize  the  notion  of  a  hybrid  system  governed  by  a  hybrid  controller  using 
an  optimal  control  framework.  We  prove  theoretical  results  that  lead  us  to  alpnthms 
for  synthesizing  such  hybrid  controllers.  In  particular,  we  prove  existence  of  optimal  and 
near  optimal  controls  and  derive  “generalized  quasi- variational  inequaUties”  that  the  asso¬ 
ciated  value  function  satisfies.  We  outline  algorithms  for  solving  these  inequalities,  based 
on  a  generalized  Bellman  equation,  impulse  control  algorithms,  and  hnear  programming. 
Several  illustrative  examples  are  solved.  The  synthesized  optimal  hybrid  controllers  verify 
engineering  intuition. 
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Abstract 

Complex  systems  typically  possess  a  hierarchical  structure,  characterized  by  continuous- 
variable  dynamics  at  the  lowest  level  and  logical  decision-making  at  the  highest.  Virtu¬ 
ally  all  control  systems  today  perform  computer-coded  checks  and  issue  logical  as  well  as 
continuous- variable  control  commands.  Such  are  “hybrid”  systems. 

IVaditionally,  the  hybrid  nature  of  these  systems  is  suppressed  by  converting  them  into 
either  purely  discrete  or  continuous  entities.  Motivated  by  real-world  problems,  we  introduce 
“hybrid  systems”  as  interacting  collections  of  dynamical  systems,  evolving  on  continuous- 
variable  state  spaces,  and  subject  to  continuous  controls  and  discrete  phenomena. 

We  identify  the  discrete  phenomena  that  arise  in  hybrid  systems  and  review  previously 
proposed  models.  We  propose  a  hybrid  control  model,  coupling  differential  equations  and 
automata,  that  encompasses  them.  Our  unified  model  is  natural  for  posing  and  solving 
hybrid  analysis  and  control  problems. 

We  discuss  topological  issues  that  arise  in  hybrid  systems  analysis.  Then  we  compare 
the  computational  capabilities  of  analog,  digital,  and  hybrid  machines  by  proposing  intu¬ 
itive  notions  of  analog  machines  simulating  digital  ones.  We  show  that  simple  continuous 
systems  possess  the  power  of  universal  computation.  Hybrid  systems  have  further  simula¬ 
tion  capabilities.  For  instance,  we  settle  the  famous  asynchronous  arbiter  problem  in  both 
continuous  and  hybrid  settings.  Further,  we  develop  analysis  tools  for  limit  cycle  existence, 
perturbation  robustness,  and  stability.  We  analyze  a  hybrid  control  system,  typically  used 
in  aircraft,  that  logically  switches  between  two  conventional  controllers.  Stability  of  such 
systems  has  previously  only  been  tested  using  extensive  simulation;  we  prove  global  asymp¬ 
totic  stability  for  a  realistic  set  of  cases.  Our  tools  demonstrate  robustness  of  this  stability 
with  respect  to  “continuation”  of  the  logical  function. 

We  systematize  the  notion  of  a  hybrid  system  governed  by  a  hybrid  controller  using 
an  optimal  control  framework.  We  prove  theoretical  results  that  lead  us  to  algorithms 
for  synthesizing  such  hybrid  controllers.  In  particular,  we  prove  existence  of  optimal  and 
near  optimal  controls  and  derive  “generalized  quasi-variational  inequalities”  that  the  asso¬ 
ciated  value  function  satisfies.  We  outline  algorithms  for  solving  these  inequalities,  based 
on  a  generalized  Bellman  equation,  impulse  control  algorithms,  and  linear  programming. 
Several  illustrative  examples  are  solved.  The  synthesized  optimal  hybrid  controllers  verify 
engineering  intuition. 
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Chapter  1 

Introduction  to  Hybrid  Systems 


In  this  chapter,  we  motivate  a  study  of  hybrid  systems  and  examine  real-world  examples 
where  they  arise.  We  explain  our  theoretical  paradigm  for  such  a  study  and  formally 
introduce  the  objects  of  that  study:  autonomous  and  controlled  hybrid  dynamical  systems. 
We  outline  the  thesis  and  summarize  its  contributions. 


§1.1  INTRODUCTION 

MOTIVATION 

Motivated  by  biology  and  a  study  of  complex  systems,  intelligent  behavior  is  typically 
associated  with  a  hierarchical  structure.  Such  a  hierarchy  exhibits  an  increase  in  reaction 
time  and  abstraction  with  increasing  level.  In  both  natural  and  engineered  systems  the 
lowest  level  is  usually  characterized  by  continuous-variable  dynamics  and  the  highest  by 
a  logical  decision-making  mechanism.  The  interaction  of  these  different  levels,  with  their 
different  types  of  information,  leads  to  a  “hybrid”  system. 

Many  complicated  control  systems  today  (e.g.,  those  for  flight  control,  manufacturing 
systems,  and  transportation)  have  vast  amounts  of  computer  code  at  their  highest  level. 
More  pervasively,  programmable  logic  controllers  are  widely  used  in  industrial  process  con¬ 
trol.  We  also  see  that  today’s  products  incorporate  logical  decision-making  into  even  the 
simplest  control  loops  (e.g.,  embedded  systems).  Thus,  virtually  all  control  systems  today 
issue  continuous-variable  controls  and  perform  logical  checks  that  determine  the  mode 
aind  hence  the  control  algorithms — the  continuous- variable  system  is  operating  under  at  any 
given  moment.  As  such,  these  “hybrid  control”  systems  offer  a  challenging  set  of  problems. 

So,  “hybrid”  systems  are  certainly  pervasive  today.  But  they  have  been  with  us  at 
least  since  the  days  of  the  relay.  Traditionally,  though,  the  hybrid  nature  of  systems  and 
controllers  has  been  suppressed  by  converting  them  into  either  purely  discrete  or  purely 
continuous  entities.  The  reason  is  that  science  and  engineering’s  formal  modeling,  analysis, 
and  control  “toolboxes”  deal  largely— and  largely  successfully— with  these  “pure”  systems. 

Engineers  have  pushed  headlong  into  the  application  areas  above.  And  the  successes 
in  flight  control  alone  attest  to  the  fact  that  it  is  possible  to  build  highly  complex,  highly 
reliable  systems.  Yet  ever  more  complex  systems  continue  to  arise  (e.g.,  flight  vehicle 
management  and  intelligent  vehicle/highway  systems).  And  the  trend  toward  embedded 
systems  is  sure  to  continue. 

It  is  time  to  focus  on  developing  formal  modeling,  analysis,  and  control  methodologies 
for  “hybrid  systems.” 
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WHAT  ARE  HYBRID  SYSTEMS? 

Generalizing  from  the  examples  above,  hybrid  systems  involve  both  continuous-valued  and 
discrete  variables.  Their  evolution  is  given  by  equations  of  motion  that  generally  depend  on 
all  variables.  In  turn  these  equations  contain  mixtures  of  logic,  discrete-valued  or  digital 
dynamics,  and  continuous-variable  or  analog  dynamics.  The  continuous  dynamics  of  such 
systems  may  be  continuous-time,  discrete-time,  or  mixed  (sampled-data),  but  is  generally 
given  by  differential  equations.  The  discrete- variable  dynamics  of  hybrid  systems  is  generally 
governed  by  a  digital  automaton,  or  input-output  transition  system  with  a  countable 
number  of  states.  The  continuous  and  discrete  dynamics  interact  at  “event”  or  “trigger” 
times  when  the  continuous  state  hits  certain  prescribed  sets  in  the  continuous  state  space. 
See  Figure  1-1. 


Figure  1-1:  Hybrid  System. 


Hybrid  control  systems  are  control  systems  that  involve  both  continuous  and  discrete 
dynamics  and  continuous  and  discrete  controls.  The  continuous  dynamics  of  such  a  system 
is  usually  modeled  by  a  controlled  vector  field  or  difference  equation.  Its  hybrid  nature  is 
expressed  by  a  dependence  on  some  discrete  phenomena,  corresponding  to  discrete  states, 
dynamics,  and  controls.  The  result  is  a  system  as  in  Figure  1-2. 
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Figure  1-2:  Hybrid  Control  System. 

Below,  we  introduce  hybrid  systems  as  interacting  collections  of  dynamical  systems, 
each  evolving  on  Continuous  state  spaces,  and  subject  to  continuous  and  discrete  controls, 
and  some  other  discrete  phenomena. 
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STUDIES  IN  HYBRID  SYSTEMS 

Research  into  such  hybrid  systems  may  be  broken  down  into  four  broad  categories: 

•  Modeling:  formulating  precise  models  that  capture  the  rich  behavior  of  hybrid  sys¬ 
tems. 

How  do  we  “fill  in  the  boxes”  in  Figures  1-1  and  1-2? 

•  Analysis:  developing  tools  for  the  simulation,  analysis,  and  verification  of  hybrid 
systems. 

How  do  we  analyze  systems  as  in  Figure  1-1? 

•  Control:  synthesizing  hybrid  controllers  — which  issue  continuous  controls  and  make 
discrete  decisions— that  achieve  certain  prescribed  safety  and  performance  goals  for 
hybrid  systems. 

How  do  we  control  a  plant  as  in  Figure  1-2  with  a  controller  as  in  Figure 

1-2? 

•  Design:  conceiving  new  schemes  and  structures  that  lead  to  easier  modeling,  verifi¬ 
cation,  and  control  of  hybrid  systems. 

In  this  thesis,  we  concentrate  on  the  first  three  categories,  in  Parts  I,  II,  and  III,  respec¬ 
tively.  Dependencies  among  the  material  is  given  in  Figure  1-3. 

Next,  we  give  a  quick  overview  of  the  thesis  and  its  contributions.  For  more  details 
consult  §§1.5-1.7,  in  which  we  discuss  modeling,  analysis,  and  control  contributions,  in 
more  depth. 

Modeling.  After  examining  the  real-world  examples  in  this  chapter,  we  identify  the  dis¬ 
crete  phenomena  that  generally  arise  in  hybrid  systems  and  give  more  examples.  Then,  we 
review  in  detail  six  previously  posed  hybrid  systems  models,  primarily  from  the  systems  and 
control  literature.  We  make  some  comparisons  which  enable  us  to  later  prove  simulation 
and  modeling  results  for  such  systems. 

Throughout,  we  concentrate  on  the  state-space  and  consider  general  hybrid  dynamical 
systems  as  an  indexed  collection  of  dynamical  systems  along  with  some  (deterministic,  or 
autonomous)  rules  for  “jumping”  among  them.  These  jumps  take  the  form  of  switching 
dynamical  systems  and/or  resetting  their  “continuous”  states.  This  jumping  generally  oc¬ 
curs  whenever  the  state  satisfies  certain  conditions,  given  by  its  membership  in  a  specified 
subset  of  the  state  space. 

Controlled  hybrid  systems  add  the  possibility  of  making  discrete  decisions  at  autonomous 
jump  times  as  well  the  ability  to  discontinuously  reset  state  variables  at  “intervention  times” 
when  the  state  satisfies  certain  conditions,  given  by  its  membership  in  another  specified 
subset  of  the  state  space.  In  general,  the  allowed  resettings  depend  on  the  state. 

We  introduce  a  hierarchy  of  such  systems  and  provide  a  taxonomy  of  them  based  on 
their  structure  and  the  discrete  phenomena  they  exhibit.  We  also  give  explicit  instructions 
for  computing  the  orbits  and  trajectories  of  general  hybrid  dynamical  systems,  including 
sufl[icient  conditions  for  existence  and  uniqueness. 

As  a  final  result  we  introduce  a  precise  model  for  hybrid  control  that  is  shown  to  en¬ 
compass  all  identified  phenomena  and  subsume  all  reviewed  models.  This  “unified”  model 
is  suitable  for  the  posing  and  solution  of  analysis  and  control  problems  in  the  sequel. 
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Figure  1-3:  Dependency  Structure  of  Thesis  Topics. 


Analysis.  We  first  discuss  topplogical  issues  that  arise  in  hybrid  systems  analysis.  In  par¬ 
ticular,  we  examine  topologies  for  achieving  continuity  of  maps  from  a  set  of  measurements 
of  continuous  dynamics  to  a  finite  set  of  input  symbols  {AD  map  of  Figure  1-1)  and  back 
again  (DA  map).  Finding  some  anomalies  in  completing  this  loop,  we  discuss  a  different 
view  of  hybrid  systems  that  can  broach  them  and  is  more  in  line  with  traditional  control  sys¬ 
tems.  The  most  widely  used  fuzzy  control  system  is  related  to  this  different  view  and  does 
not  possess'  these  anomalies.  Indeed,  we  show  that  fuzzy  control  leads  to  continuous  maps 
(from  measurements  to  controls)  and  that  all  such  continuous  maps  may  be  implemented 
via  fuzzy  control. 

Then  we  compare  the  simulation  and  computational  capabilities  of  analog,  digital,  and 
hybrid  Tnarhinps  We  accomplish  this  by  proposing  several  new,  but  intuitive,  notions  of 
simulation  of  a  digital  machine  by  an  analog  one.  We  show  that  even  simple  continuous 
systems,  namely  smooth  ODEs  in  R^,  possess  the  power  of  universal  computation.  And  our 
simulation  definitions  do  not  require  infinite  precision  or  precise  timing.  Hybrid  systems 
have  further  simulation  capabilities.  We  contrast  hybrid  and  continuous  systems  by  solv¬ 
ing  the  well-known  arbiter  problem  in  both  a  differential  equations  and  a  hybrid  systems 
framework: 
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•  You  cannot  build  an  arbiter  with  a  set  of  Lipschitz  ODEs  with  continuous  output 
map. 

•  There  is  a  simple  (2-discrete-state,  no  continuous-state-jump)  hybrid  system — whose 
component  ODEs  are  Lipschitz  with  continuous  output  map — that  meets  arbiter  spec¬ 
ifications.  Each  reviewed  model  can  implement  it. 

Then  we  develop  some  tools  for  the  analysis  of  hybrid  systems.  In  particular,  we  develop 
two  tools  for  stability  of  such  systems: 

•  a  new  one  which  we  call  multiple  Lyapunov  functions  for  studying  Lyapunov  stability, 

•  iterated  function  systems  (IFS)  as  a  tool  for  Lagrange  stability  and  positive  invariance. 

Other  tools,  such  as  an  extension  of  Bendbcson’s  Theorem  for  detecting  limit  cycles  of  certain 
hybrid  systems  and  a  robustness  lemma  for  differential  equations,  are  also  developed.  These 
tools  have  applications  beyond  the  scope  of  hybrid  systems. 

Finally,  our  attention  focuses  on  example  systems  and  their  analysis.  The  example 
systems,  denoted  max  systems,  arise  from  a  realistic  aircraft  control  problem  which  logically 
switches  between  two  controllers  (one  for  tracking  and  one  for  regulation  about  a  fixed  angle 
of  attack)  in  order  to  achieve  both  reasonable  performance  and  safety.  While  stability  of 
such  systems  has  previously  only  been  examined  using  extensive  simulation  [134],  we  are 
able  to  prove  global  asymptotic  stability  for  a  realistic  class  of  cases.  Using  our  robustness 
lemma  to  compare  ODE  solutions,  we  extend  the  result  to  a  class  of  “continuations”  of  max 
systems,  which  dynamically  smooth  the  logical  nonlinearity. 

Control.  We  systematize  the  notion  of  a  hybrid  system  governed  by  a  hybrid  controller 
using  an  optimal  control  framework.  We  prove  theoretical  results  that  lead  us  to  algorithms 
for  synthesizing  such  hybrid  controllers. 

In  particular,  we  define  an  optimal  control  problem  in  our  unified  hybrid  control  firame- 
work  and  derives  some  theoretical  results.  The  problem,  and  all  assumptions  used  in  obtain¬ 
ing  the  remaining  results,  are  expressly  stated.  Further,  the  necessity  of  these  assumptions 
or  ones  like  them — is  demonstrated.  The  main  results  are  as  follows: 

•  We  prove  the  existence  of  optimal  and  near  optimal  controls. 

•  We  derive  “generalized  quasi-variational  inequalities”  (GQVIs)  that  the  associated 
value  function  is  expected  to  satisfy. 

Using  the  GQVIs  as  a  starting  point,  we  concentrate  on  algorithms  for  solving  hybrid 
control  problems.  Our  unified  view  led  to  the  concept  of  examining  a  “generalized  Bellman 
equation.”  We  also  draw  explicit  relations  with  impulse  control  of  piecewise-deterministic 
processes.  Four  algorithmic  approaches  are  outlined: 

•  an  explicit  boundary-value  algorithm, 

•  generalized  value  iteration  and  policy  iteration, 

•  modified  impulse  control  algorithms, 

•  linear  programming. 

Finally,  three  illustrative  examples  are  solved  in  our  firamework.  We  consider 
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•  a  hysteresis  system  that  exhibits  autonomous  switching  and  has  a  continuous  control; 

•  a  satellite  station-keeping  problem  involving  controlled  switching; 

•  a  transmission  problem  with  continuous  accelerator  input  and  discrete  gear-shift  po¬ 
sition.  In  each  case,  the  optimal  controls  produced  verify  engineering  intuition. 

Conclusion.  §12  concludes  with  a  summary  of  contributions  and  a  list  of  some  open  issues 
and  future  directions,  including  some  words  on  Design. 

AN  INTRODUCTION  ... 

There  has  been  much  recent  interest  in  studying  hybrid  systems  [3,  5,  9,  32,  38,  56,  63,  64, 
65,  66,  114,  120].  This  chapter  is  both  an  introduction  to  that  field  and  to  this  thesis.  It  is 
organized  as  follows. 

We  have  already  informally  defined  hybrid  systems.  In  the  next  section,^  §1.2,  we 
provide  real-world  examples  of  hybrid  systems.  These  serve  the  dual  purpose  of  illustration 
and  motivation.  Next,  we  give  a  short  history  and  examine  approaches  to  the  studies  of 
hybrid  systems.  One  is  the  paradigm  adopted  here:  a  hybrid  system  as  an  interacting 
collection  of  dynamical  systems. 

Thus,  in  §1.4  we  are  led  to  review  the  notion  of  a  general  dynamical  system.  This  sets 
the  stage  for  us  to  formally  define  the  objects  of  our  study:  hybrid  dynamical  systems.  Since 
we  are  interested  in  control,  we  introduce  both  autonomous  and  controlled  versions. 

We  have  already  outlined  our  contributions  above.  In  §§1.5— 1.7,  we  consecutively  discuss 
modeling,  analysis,  and  control  contributions  in  more  depth. 

Throughout  the  thesis  we  assume  some  familiarity  with  control  theory  [96,  133],  differ¬ 
ential  equations  [73],  automata  theory  [20,  76],  and  topology  [62,  75,  113].  Some  review  of 
notation  arid  material  is  done  in  §2.1  and  §A.  A  majority  of  the  notation  is  collected  in  the 
Symbol  Index,  pp.  193-194.  A  general  Index  is  also  provided  for  convenience. 


^The  symbol  §  is  used  to  cross-reference  thesis  sections. 
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§1.2  REAL-WORLD  EXAMPLES  OF  HYBRID  SYSTEMS 

The  protot5rpical  hybrid  systems  axe  digital  controllers,  computers,  and  subsystems  mod¬ 
eled  as  finite  automata  coupled  with  controllers  and  plants  modeled  by  partial  or  ordinary 
differential  equations  or  difference  equations.  Thus,  such  systems  arise  whenever  one  mixes 
logical  decision-making  with  the  generation  of  continuous  control  laws.  More  specifically, 
real-world  examples  of  hybrid  systems  include 

•  systems  with  relays,  switches,  and  hysteresis  [135,  152], 

•  computer  disk  drives  [65], 

•  transmissions,  stepper  motors,  and  other  motion  controllers  [38], 

•  constrained  robotic  systems  [9], 

•  intelligent  vehicle/highway  systems  (IVHS)  [64,  140], 

•  modern  flexible  manufacturing  and  flight  control  systems  [80,  103]. 

Other  important  application  areas  for  hybrid  systems  theory  include  embedded  systems  and 
analog/digital  circuit  co-design  and  verification. 

We  now  briefly  examine  each  of  the  above  examples  in  more  detail  in  turn. 

Systems  with  Switches  and  Relays.  Physical  systems  with  switches  and  relays  can  be 
naturally  modeled  as  hybrid  systems.  Sometimes,  the  dynamics  may  be  considered  merely 
discontinuous,  such  as  in  a  blown  fuse.  In  many  cases  of  interest,  however,  the  switching 
mechanism  has  some  hysteresis,  yielding  a  discrete  state  on  which  the  dynamics  depends. 
This  situation  is  depicted  by  the  multi-valued  function  H  shown  in  Figure  1-4. 
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Figure  1-4;  Hysteresis  Function. 

Suppose  the  function  H  models  the  hysteretic  behavior  of  a  thermostat.  We  may  model 
a  thermostatically  controlled  room  as  follows 

i  =  /(x,ff(x-xo)),  (1-1) 

where  x  and  xq  denote  room  and  desired  temperature,  respectively.  The  function  /  denotes 
dynamics  of  temperature,  which  depends  on  the  current  .temperature  and  whether  the 
furnace  is  switched  On  or  Off.  Note  that  this  system  is  not  just  a  differential  equation  whose 
right-hand  side  is  piecewise  continuous.  There  is  “memory”  in  the  system,  which  affects  the 
value  of  the  vector  field.  Indeed,  such  a  system  naturally  has  a  finite  automaton  associated 
with  the  hysteresis  function  H,  as  pictured  in  Figure  1-5.  Notice  that,  for  example,  the 
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discrete  state  changes  from  +1  to  —1  when  the  continuous  state  enters  the  set  {x  >  A}. 
That  is,  the  event  of  x  attaining  a  value  greater  than  or  equal  to  A  triggers  the  discrete  or 
phase  transition  of  the  underlying  automaton. 
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Figure  1-5:  Finite  Automaton  Associated  with  Hysteresis  Function. 


Disk  Drive.  A  computer  disk  drive  may  be  modeled  as  a  black  box  that  receives  exter¬ 
nal  Read  commands  and  outputs  bytes.  The  action  of  the  disk  drive  is  governed  by  the 
differential  (or  difference  equations)  modeling  the  dynamic  behavior  of  the  disk,  spindle, 
disk  arm,  and  motors.  The  drive  receives  symbolic  inputs  of  disk  sectors  and  locations;  it 
transmits  symbolic  outputs  corresponding  to  the  bytes  read.  It  may  also  receive  symbolic 
commands  like  Reinitialize  and  transmit  symbolic  outputs  like  ReadError.  See  Figure  1-6, 
which  is  constructed  from  the  description  in  [65]. 
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Figure  1-6:  Finite  State  Machine  Associated  with  Disk  Drive  Activities. 
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Transmission.  An  automobile  transmission  system  takes  the  continuous  inputs  of  acceler¬ 
ator  position  and  engine  RPM  and  the  discrete  input  of  gear  position  and  translates  them 
into  motion  of  the  vehicle.  Suppose  one  is  designing  a  cruise  control  system  that  accelerates 
and  decelerates  under  different  profiles.  The  desired  profile  is  chosen  depending  on  sensor 
readings  (e.g.,  continuous  reading  of  elevation,  discrete  coding  of  road  condition,  etc.).  In 
such  a  case,  we  are  to  design  a  control  system  with  both  continuous  and  discrete  states  and 
controls.  See  Example  3.4,  p.  54. 

Hopping  Robot.  Interesting  examples  of  hybrid  systems  are  constrained  robotic  systems. 
In  particular,  consider  the  hopping  robots  of  Marc  Raibert  of  MIT  [122].  The  dynamics 
of  these  devices  are  governed  by  gravity,  as  well  as  the  forces  generated  by  passive  and 
active  (pneumatic)  springs.  The  dynamics  change  abruptly  at  certain  event  times,  and  fall 
into  distinct  phases:  Flight,  Compression,  Thrust,  and  Decompression.  See  Figure  1-7.  In 
fact,  Raibert  has  built  controllers  for  these  machines  that  embed  a  finite  state  machine  that 
transitions  according  to  these  detected  phases.  For  instance,  the  transition  from  Flight  to 
Compression  occurs  when  touchdown  is  detected;  that  from  Decompression  to  Flight  upon 
liftoff.  Thus,  finite  automata  and  differential  equations  naturally  interact  in  such  devices 
and  their  controllers. 
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Figure  1-7:  The  dynamic  phases  of  Raibert’s  hopping  robot.  Reproduced  firom  [9]. 
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IVHS.  A  more  complicated  example  of  a  hybrid  system  arises  in  the  control  structures 
for  so-called  intelligent  vehicle  and  highway  systems  (IVHS)  [106,  140].  The  basic  goal 
of  one  such  system  is  to  increase  highway  throughput  by  means  of  a  technique  known  as 
platooning.  A  platoon  is  a  group  of  between,  say,  one  and  twenty  vehicles  traveling  closely 
together  in  a  highway  lane  at  high  speeds.  To  ensure  safety — and  proper  formation  and 
dissolution  of  structured  platoons  from  the  “free  agents”  of  single  vehicles— requires  a  bit 
of  control  effort!  As  in  the  theory  of  communication  networks,  researchers  have  broken  this 
control  task  into  layers  [140].  See  Figure  1-8. 


Figure  1-8:  IVHS  Control  System  Architecture. 

Protocols  for  basic  maneuvers  such  as  Merge,  Split,  and  ChangeLane  have  been  proposed 
in  terms  of  finite  state  machines.  More  conventional  controllers  govern  the  engines  and 
brakes  of  individual  vehicles.  Clearly,  the  system  is  hybrid.  Each  vehicle  has  a  state 
determined  by: 

•  continuous  variables,  such  as  velocity,  engine  RPM,  distance  to  car  ahead, 

•  the  finite  state  of  its  protocol-enacting  automata. 

The  more  conventional  controllers  can  be  analyzed  for  good  performance  using  methods 
in  control  theory.  The  protocol  designs  can  be  verified  using  tools  such  as  AT&T’s  COSPAN 
[72].  See  Figure  1-9.  The  challenge  with  IVHS  is  to  analyze  the  interconnected  system  as 
a  whole:  One  has  to  verify  (by  proof  or  simulation)  that  the  vehicles  enact  the  protocol 
correctly  and  safely,  under  a  range  of  dynamics  conditions,  for  each  of  the  possible  product 
of  finite  states,  for  a  wide  scope  of  scenarios. 
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Figure  1-9;  COSPAN  specification  of  IVHS  Merge  protocol.  Reproduced  from  [140]. 


Complex  Systems.  In  the  case  of  modern  fiexible  manufacturing  or  flight  control  systems, 
there  are  typically  many  control  subsystems  with  many  control  modes  for  each  subsystem. 
The  control  subsystems  that  are  active  and  the  modes  that  are  enforced  at  any  given 
time  is  usually  determined  by  a  computer  program.  This  computer  program  is  a  complex 
dynamical  system  in  its  own  right  and  may  be  modeled  as  a  finite  automaton,  pushdown 
automaton,  Petri  net,  Turing  machine,  etc.  The  “transitions”  of  the  computer  program 
axe  not  entirely  independent  of  the  physical  system;  many  depend  on  the  logical  truth  of 
statements  concerning  the  continuous  values  of  the  physical  variables.  Since  the  logical  state 
of  the  computer  program  determines  which  control  mode  is  in  use,  the  evolution  of  states 
of  the  physical  system  is  likewise  influenced  by  the  values  of  these  logical  variables.  In  real- 
world  systems,  this  interaction  is  complicated  by  the  fact  that  there  is  not  a  strict  dichotomy 
between  discrete  or  logical  components  and  continuous  or  physical  components.  The  entire 
system  is  made  up  of  subsystems  that  we  choose  to  model  as  discrete,  continuous,  or  hybrid. 
Figure  1-10  depicts  a  course-grained  model  of  a  modem  flight  vehicle  management  system 
(FVMS)  proposed  for  the  High-Speed  Civil  Transport  [80,  103]. 

Interactions  among  the  subsystems  of  a  FVMS  are  typically  so  complicated  that  the 
only  course  of  action  in  analyzing  such  systems  is  exhaustive  simulation.  This  is  currently 
done  nsing  programs  such  as  Statemate  [69,  70,  71,  81].  See  Figure  1-11.  The  “statechart” 
pictured  represents  a  simplified  mode  controller.  It  switches  from  the  nominal  mode  of 
“pitch  controlling  climb,  throttle  controlling  speed”  to  modes  of  “pitch  controlling  speed, 
riimh  clamped  high/low”  when  the  throttle  exceeds  upper  and  lower  limits,  respectively. 
There  is  also  a  “manual”  mode,  corresponding  to  the  autopilot’s  being  turned  off. 

In  §9,  we  analyze  a  closely  related  aircraft  control  system,  termed  the  moi  system, 
which  switches  between  two  different  controllers  in  order  to  achieve  a  good  tradeoff  between 
performance  and  safety  constraints.  See  Figure  9-2,  p.  139. 


VEHICLE  MANAGEMENT  SYSTEM 
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Figure  1-10:  Flight  Vehicle  Management  System  architecture.  Reproduced  from  [80]. 
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Figure  1-11:  Statechaxt  Description  of  a  Simplified  FVMS  Mode  Controller. 


§1.3  STUDIES  IN  HYBRID  SYSTEMS 

§1.3.1  HISTORY  IN  BRIEF 

Hybrid  systems  axe  certainly  pervasive  today.  But  they  have  been  with  us  at  least  since 
the  days  of  the  relay.  The  earliest  direct  reference  we  know  of  is  the  visionary  work  of  Hans 
Witsenhausen  from  MIT,  who  formulated  a  class  of  hybrid-state,  continuous-time  dynamic 
systems  and  examined  an  optimal  control  problem  [152,  §3.3,  §10.5].  This  was  followed  by 
others,  like  Pavlidis,  who  studied  stability  of  systems  with  impulses  via  Lyapunov  functions 
[118,  §8.8].  Other  early  work  on  hybrid  systems  also  came  from  MIT  [83, 137, 151],  where  the 
interest  was  in  finite  state  controllers.  Ezzine  and  Haddad  examined  stability,  controllability, 
and  observability  of  a  restricted  class  of  switched  linear  systems  [60,  §2.2.2].  Motivated  by 
an  interest  in  systems  with  hysteresis,  Tavernini  produced  a  precise  hybrid  systems  model 
and  proved  results  on  initial-value  problems  and  their  numerical  approximations  [135,  §3.4]. 

In  control  theory,  there  has  certainly  been  a  lot  of  related  work  in  the  past,  includ¬ 
ing  variable  structure  systems,  jump  linear  systems,  systems  with  impulse  effect,  impulse 
control,  and  piecewise  deterministic  processes.  These  are  quickly  reviewed  in  §2.2.  In  com¬ 
puter  science,  there  has  been  a  successive  build-up  in  the  large  formal  verification  literature 
[15,  54,  68,  84,  99,  146]  toward  verification  of  systems  that  include  both  continuous  and 
discrete  variables  [3,  16,  66,  98,  120]. 

Recently,  we  have  witnessed  a  resurgence  in  examining  quantization  effects  [55,  89, 123, 
128]  and  a  heightened  interest  in  analog  computation  [19,  27,  37,  46,  131,  §7].  Finally,  there 
has  also  been  recent  progress  in  analyzing  switched  [30,  93,  119,  149,  §4.5,  §8],  hierarchical 
[40,  146],  and  discretely-controlled  continuous- variable  systems  [41,  90,  112,  110,  111]. 

Hybrid  systems  have  just  started  to  be  addressed  more  wholeheartedly  by  the  control 
community  [3,  5,  9,  32,  38,  56,  63,  64,  65,  66, 114, 120].  Computer  scientists  have  also  begun 
to  attack  this  area  [3,  66,  120]. 
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§1.3.2  PARADIGMS 

We  see  four  basic  paradigms  for  the  study  of  hybrid  systems:  aggregation,  continuation, 
automatization,  and  systemization.  The  first  two  approaches  deal  with  the  different  sides 
analog  and  digital — of  hybrid  systems.  They  attempt  to  suppress  the  hybrid  nature  of  the 
system  by  converting  it  into  a  pmely  discrete  or  purely  continuous  one,  respectively.  The 
last  two  approaches  are  both  more  general  and  potentially  more  powerful.  Under  them, 
a  hybrid  system  is  seen  directly  as  an  interacting  set  of  automata  or  dynamical  systems; 
they  complement  the  input-output  and  state-space  paradigms,  respectively,  of  both  control 
theory  and  computer  science.  More  specifically,  the  approaches  are  as  follows. 

1.  Aggregation  That  is,  suppress  the  continuous  dynamics  so  that  the  hybrid  system  is 
a  finite  automaton  or  discrete-event  dynamical  system  [74].  This  is  the  approach  most 
often  taken  in  the  literature,  e.g.,  [5].  The  drawback  of  this  approach  is  three-fold. 

•  NondeteTTninism,  i.e.,  one  usually  obtains  a  nondeterministic  automaton.  This 
was  noted  by  Antsaklis,  Stiver,  and  Lemmon  [5].  Also  cf.  Hsu  s  cell-to-cell  map¬ 
ping  [78]. 

•  Nonexistence,  i.e.,  even  if  clever  constructions  are  used,  no  finite  automaton  may 
exist  that  captures  the  combined  behavior  [63]. 

•  Partition  Problem.  It  appears  a  conceptually  deep  problem  to  determine  when 
there  exist  partitions  of  just  a  continuous  system  such  that  its  dynamics  is  cap)- 
tured  by  a  meaningful  finite  automaton.  “Meaningful,”  since  we  note  that  every 
system  is  homomorphic  to  one  with  a  single  equilibrium  point  [129,  §2.1.2].  The 
answer  thus  depends  on  the  dynamical  behavior  one  is  interested  in  capturing 
and  the  questions  one  is  asking.  Readers  interested  in  pursuing  this  topic  should 
consult  work  on  analog  simulation  of  digital  machines  [27,  31,  36]  as  well  as  the 
more  recent  work  of  Prof.  Roger  Brockett  [39]. 

The  aggregation  program  has  been  fully  carried  out  so  far  only  under  strong  assump¬ 
tions  on  the  hybrid  system  [1,  63]. 

2.  Continuation,  the  complement  of  aggregation,  that  is,  suppress  the  discrete  dynam¬ 
ics  so  that  the  hybrid  system  becomes  a  differential  equation.  This  original  idea  of 
Prof.  Sanjoy  Mitter  and  the  author  is  to  convert  hybrid  models  into  purely  continuous 
ones— modeled  by  differential  equations— using  differential  equations  that  simulate  fi¬ 
nite  automata.  In  this  familiar,  unified  realm  one  could  answer  questions  of  stability, 
controllability,  and  observability,  converting  them  back  to  the  original  model  by  taking 
a  “singular  limit.”  For  instance,  one  would  like  tools  that  allow  one  to  conclude  the 
following:  if  a  “sufficiently  close”  continuation  of  a  system  is  stable,  then  the  original 
system  is  stable.  Such  a  program  is  possible  in  light  of  the  existence  of  simple  con¬ 
tinuations  of  finite  automata  [31,  36]  and  pushdown  automata  and  Turing  machines 
[31].  The  drawback  of  this  approach  is  three-fold. 

•  Arbitrariness,  i.e.,  how  one  accomplishes  the  continuation  is  largely  arbitrary. 
For  example,  to  interpolate  or  “simulate”  the  step-by-step  behavior  of  a  finite 
automaton  Brockett  used  his  double-bracket  equations  [38]  and  the  author  used 
stable linear  equations  [22,  31].  In  certain  cases  this  freedoni  is  an  advantage 
[§7].  However,  care  must  be  taken  to  insiu-e  that  the  dynamics  used  does  not 
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introduce  spurious  behavior  (like  unwanted  equilibria)  or  that  it  itself  is  not  hard 
to  analyze  or  predict. 

•  Hiding  Complexity.  One  cannot  generally  get  rid  of  the  underlying  discrete 
dynamics,  i.e.,  the  complexity  is  merely  hidden  in  the  right-hand  side  of  the 
continuation  differential  equations  [27]. 

•  Artificiality.  It  can  lead  to  a  possibly  unnatural  analytical  loop  of  going  from 
discrete  to  continuous  and  back  to  discrete.  Cf.  Chen’s  recent  results  in  stochastic 
approximation  vis-a-vis  Kushner’s  [42,  43]. 

The  combination  of  these  points  has  been  borne  out  by  some  experience:  it  can  be 
easier  to  examine  the  mixed  discrete-continuous  system.  Cf.  our  analysis  of  a  switched 
aircraft  controller  [28,  §9]  and  Megretsky’s  recent  analysis  of  a  relay  system  [102]. 

3.  Automatization  or  automata  approach.  Treat  the  constituent  systems  as  a  net¬ 
work  of  interacting  automata  [114,  p.  325].  The  focus  is  on  the  input-output  or  lan¬ 
guage  behavior  [20,  76].  The  language  view  has  been  largely  taken  in  the  computer 
science  literature  in  extending  the  dynamical  behavior  of  finite  automata  incremen¬ 
tally  toward  full  hybrid  systems  (see  [1,  66]  for  background). 

Automatization  was  pioneered  in  full  generality  by  Nerode  and  Kohn  [114].  The 
viewpoint  is  that  systems,  whether  analog  or  digital,  are  automata.  As  long  as  there 
is  compatibility  between  output  and  input  alphabets,  links  between  automata  can  be 
established.  However,  there  is  still  the  notion  of  “reconciling  different  time  scales” 
[114,  p.  325].  For  instance,  a  finite  automaton  receives  symbols  in  abstract  time, 
whereas  a  differential  equation  receives  inputs  in  “real  time.”  This  reconciliation  can 
take  place  by  either  of  the  following: 

•  forcing  synchronization  at  regular  sampling  instants  [114,  p.  333], 

•  synchronizing  the  digital  automaton  to  advance  at  event  times  when  its  input 
symbols  change  [114,  §3.2.5]. 

For  hybrid  systems  of  interest,  the  latter  mechanism  appears  more  useful.  It  has  been 
used  in  many  hybrid  systems  models,  e.g.,  [5,  38,  §§3. 7-3.8].  It  is  reviewed  in  §3.6. 

The  automata  approach  has  been  taken  most  fruitfully  by  Deshpande  [56]. 

4.  Systemization  or  systems  approach.  Treat  the  constituent  systems  as  interacting 
dynamical  systems  [129,  §1.4,  §2.1.2].  The  focus  is  on  the  state-space  [117].  The 
state^pace  view  has  been  taken  most  profitably  in  the  work  of  Witsenhausen  [152] 
and  Tavernini  [135]. 

Systemization  is  developed  in  full  generality  in  this  thesis.  The  viewpoint  is  that 
systems,  whether  analog  or  digital,  are  dynamical  systems.  As  long  as  there  is  com¬ 
patibility  at  switching  times  when  the  behavior  of  a  system  changes  in  response  to 
a  logical  decision  or  event  occurrence,  links  between  these  dynamical  systems  can  be 
established.  Again,  there  is  still  the  notion  of  reconciling  dynamical  systems  with 
different  time  scales  (i.e.,  transition  semigroups).  For  instance,  a  finite  automaton  ab¬ 
stractly  evolves  on  the  positive  integers  (or  on  the  free  monoid  generated  by  its  input 
alphabet),  whereas  a  differential  equation  evolves  on  the  reals.  This  reconciliation  can 
take  place  by  either  or  both  of  the  following: 
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•  sequentially  synchronizing  the  dynamical  systems  at  event  times  when  their 
states  enter  prescribed  sets, 

•  forcing  uniform  semigroup  structure  via  “timing  maps.” 

Both  approaches  are  introduced  here,  but  the  concentration  is  on  the  former. 

Systemization  is  established  in  our  formulation  of  hybrid  dynamical  systems  below 
[§1.4,§5].  It  is  used  in  examining  complexity  and  simulation  capabilities  of  hybrid 
systems  [31,  §7],  analyzing  the  stability  of  hybrid  systems  [28,  30,  §§8-9],  and  in 
establishing  the  first  comprehensive  state-space  paradigm  for  the  control  of  hybrid 
systems  [32,  §10). 

Note.  A  different  approach  to  the  control  of  hybrid  systems  has  been  pursued  by 
Kohn  and  Nerode  [114,  Appendix  II],  in  which  the  discrete  portion  of  the  dynamics  is 
itself  designed  as  a  realizable  implementation  (that  is  a  sufficient  approximation  of)  some 
continuous  controller.  We  call  this  hybridization. 

We  are  not  really  interested  in  such  questions  in  this  thesis.  Instead,  we  wish  to  view 
both  plant  and  controller  as  hybrid  entities.  That  is,  both  are  of  the  form  of  Figure  1-2.  We 
are  motivated  to  this  view  by  the  examples  above,  such  as  FVMS.  Recall  Figure  1-10. 


§1.4  HYBRID  DYNAMICAL  SYSTEMS 

DYNAMICAL  SYSTEMS 

The  notion  of  dynamical  system  has  a  long  history  as  an  important  conceptual  tool  in 
science  and  engineering  [6,  67,  73,  96,  117,  133].  It  is  the  foundation  of  our  formulation  of 
hybrid  dynamical  systems.  We  review  it  and  some  refinements  useful  in  modeling,  analysis, 
and  control  below,  which  is  condensed  from  §2.1.2. 

Briefly,  a  dynamical  system  [129]  is  a  system 

s  =  [x,r,</)], 

where  X  is  an  arbitrary  topological  space,  the  state  space  of  S.  The  transition  semi¬ 
group  r  is  a  topological  semigroup  with  identity.  The  (extended)  transition  map 
<j)  :  X  xT  ^  X  is  &  continuous  function  satisfying  the  identity  and  semigroup  proper¬ 
ties  [§2.1.2].  A  transition  system  is  a  dynamical  system  as  above,  except  that  (f)  need  not 
be  continuous. 

Examples  of  dynamical  systems  abound,  including  autonomous  ODEs,  autonomous  dif¬ 
ference  equations,  finite  automata,  pushdown  automata,  Turing  machines,  Petri  nets,  etc. 
As  seen  from  these  examples,  both  digital  and  analog  systems  can  be  viewed  in  this  formal¬ 
ism.  The  utility  of  this  has  been  noted  since  the  earliest  days  of  control  theory  [96,  117]. 
We  will  also  denote  by  dynamical  system  the  system 

s  =  [A:,r,/], 

where  X  and  F  are  as  above,  but  the  tremsition  function  /  is  the  generator  of  the 
extended  transition  function  </>. 
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EXAMPLES.  In  the  case  of  T  =  Z,  /  :  X  -)•  X  is  given  hy  f  =  <(>{■,  1).  In  the  case  of  T  =  R, 
f  :  X  TX  is  given  by  the  vector  fields 

/(*)=  . 


We  may  also  refine  the  above  concept  by  introducing  dynamical  systems  with  initial  and 
final  states,  input  and  output,  and  timing.  See  §2.1.2. 

NOTE.  Timing  maps  provide  the  aforementioned  mechanism  for  reconciling  different  “time 
scales,”  by  giving  a  uniform  meaning  to  different  transition  semigroups  in  a  hybrid  system. 
This  is  made  clear  in  §4. 


ON  TO  HYBRID  ... 

Briefly,  a  hybrid  dynamical  system  is  an  indexed  collection  of  dynamical  systems  along 
with  some  map  for  “jumping”  among  them  (switching  dynamical  system  and/or  resetting 
the  state).  This  jumping  occurs  whenever  the  state  satisfies  certain  conditions,  given  by 
its  membership  in  a  specified  subset  of  the  state  space.  Hence,  the  entire  system  can  be 
thought  of  as  a  sequential  patching  together  of  dynamical  systems  with  initial  and  final 
states,  the  jumps  performing  a  reset  to  a  (generally  different)  initial  state  of  a  (generally 
different)  dynamical  system  whenever  a  final  state  is  reached. 

More  formally,  a  general  hybrid  dynamical  system  (GHDS)  is  a  system 

H  =  [Q,S,A,G], 


with  its  constituent  parts  defined  as  follows. 

•  Q  is  the  set  of  index  states,  also  referred  to  as  discrete  states. 

•  S  =  is  the  collection  of  constituent  dynamical  systems,  where  each  E,  = 

[Xg,  r„  <^g]  (or  S,  =  [Xg,  Tg,  fg])  Is  a  dynamical  system  as  above. 

Here,  the  Xg  are  the  continuous  state  spaces  and  <?!),  (or  fg)  are  called  the  contin¬ 
uous  dynamics. 

•  A  =  Ag  C  Xg  for  each  q£  Q,  is  the  collection  of  autonomous  jump  sets. 

•  G  =  {Gg}geQ,  where  G,  :  A,  -4  x  {?>>  collection  of  (autonomous) 

jump  transition  maps. 

These  are  also  said  to  represent  the  discrete  dynamics  of  the  hybrid  dynamical 
system. 

Thus,  5  =  U,€(3  Xg  X  {g}  is  the  hybrid  state  space  of  H.  For  convenience,  we  use 
the  following  shorthand.  Sg  =  Xg  x  {q}  and  A  =  UgeQ  ^  {?}  autonomous  jump 

set.  G  :  A-^S  is  the  autonomous  jump  transition  map,  constructed  componentwise  in  the 
obvious  way.  The  jump  destination  sets  D  =  {!>,}, e<3  are  given  by  Dg  =  7ri[G(A)  n  Sg], 
where  TTj  is  projection  onto  the  ith  coordinate.  The  switching  or  transition  manifolds. 
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Mq^p  C  Aq  are  given  by  Mq^p  =  Dp),  i*e.,  the  set  of  states  from  which  transitions 

from  index  q  to  index  p  can  occur. 

A  GHDS  can  be  pictured  as  an  automaton  as  in  Figure  1-12.  There,  each  node  is  a 
constituent  dynamical  system,  with  the  index  the  name  of  the  node.  Each  edge  represents 
a  possible  transition  between  constituent  systems,  labeled  by  the  appropriate  condition  for 
the  transition’s  being  “enabled”  and  the  update  of  the  continuous  state  (cf.  [69]).  The 
notation  ! [condition]  denotes  that  the  transition  must  be  taken  when  enabled. 


Figure  1-12:  Automaton  Associated  with  GHDS. 

Roughly,^  the  dynamics  of  the  GHDS  H  are  as  follows.  The  system  is  assumed  to  start 
in  some  hybrid  state  in  S\A,  say  sq  =  i^OiQo)-  It  evolves  according  to  (f)qQ{xo,-)  until  the 
state  enters — if  ever — at  the  point  =  (xj  ,  qo).  At  this  time  it  is  instantly  transferred 
according  to  transition  map  to  Gqq{xi)  =  {xi,qi)  =  su  from  which  the  process  continues. 
See  Figure  1-13. 

NOTES. 

1.  The  case  |(5|  =  landA  =  0isa  single  dynamical  system. 

2.  The  case  1Q|  finite,  each  X,  a  subset  of  M” ,  and  each  T,  =  R  largely  corresponds  to 
the  usual  notion  of  a  hybrid  system,  viz.  a  coupling  of  finite  automata  and  differential 
equations  [31,  32,  66].  The  two  are  coupled  at  “event  times”  when  the  continuous 
state  hits  certain  boundaries,  prescribed  by  the  sets  Aq. 

3.  Nondeterminism  may  be  added  in  the  obvious  way,  i.e.,  by  allowing  the  possibility 
that  “enabled  transitions”  need  not  be  taken  and  letting  A  be  set-valued.  Also,  see 
the  note  below. 

4.  Other  refinements  can  be  made.  See  §4.1. 

In  this  thesis,  a  hybrid  dynamical  system,  or  simply  hybrid  system,  is  defined  as 
follows: 

Definition  1.1  A  hybrid  system  is  a  general  hybrid  dynamical  system  with  Q  countable, 
and  with  F,  =  R  {or  14)  and  X,  C  R*’,  d,  €  R+,  for  all  qeQ.  In  the  notation  above,  it 


^We  make  more  precise  statements  in  §4.3. 
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Figure  1-13:  Example  dynamics  of  general  hybrid  dynamical  system. 
may  be  written  as 

[Q,[{X,}geQ,l,{/,},6Q],A,Gj 
where  fq  is  a  vector  field  on  Xg  C  . 

...AND  TO  HYBRID  CONTROL 

A  controlled  general  hybrid  dynamical  system  (GCHDS)  is  a  system 

i/c  =  [Q,S,A,G,V,C,F], 
with  its  constituent  parts  defined  as  follows. 

•  Q,  A,  and  S  are  defined  as  above. 

•  S  =  is  the  collection  of  controlled  dynamical  systems,  where  each  S,  = 

[Xg,rq,fg,Ug]  (or  Eg  =  [X,, E,, f/,] )  Is  a  controlled  dynamical  system  as  above 
with  ^extended)  transition  map  parameterized  by  control  set  Ug. 

•  G  =  where  G,  :  A,  x  F,  -4  5  is  the  autononious  jump  transition 

map,  parameterized  by  the  transition  control  set  Vg,  a  subset  of  the  collection 
V  = 

•  C  =  {Cq}geQ,  Cg  C  Xg,  is  the  collection  of  controlled  jump  sets. 

^  •p  =  {Fg}g^Q,  where  E,  :  G,  -4  2^,  is  the  collection  of  controlled  jump  destination 
maps. 

As  shorthand,  G,  G,  F  may  be  defined  as  above.  Likewise,  jump  destination  sets  Da  and 
Dc  may  be  defined.  In  this  case,  D  =  Da  U  Dc- 
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Again,  a  GCHDS  has  an  automaton  representation.  See  Figure  1-14  There,  the  notation 
? [condition]  denotes  an  enabled  transition  that  may  be  taken  on  command;  means 
reassignment  to  some  value  in  the  given  set. 


Figure  1-14:  Automaton  Associated  with  GCHDS. 


Roughly,  the  dynamics  of  He  are  as  follows.  The  system  is  assumed  to  start  in  some 
hybrid  state  in  S\A,  say  so  =  i^o^Qo)’  It  evolves  according  to  (pqQ{',',u)  until  the  state 
enters — if  ever — either  Aq^  or  Cq^  at  the  point  Sj  =  (a:^  )9o)-  If  it  enters  Aq^,  then  it  must 
be  transferred  according  to  transition  map  v)  for  some  chosen  v  €  Vq^.  If  it  enters 

C,o,  then  we  may  choose  to  jump  and,  if  so,  we  may  choose  the  destination  to  be  any 
point  in  In  either  case,  we  arrive  at  a  point  si  =  (xi,9i)  from  which  the  process 

continues.  See  Figure  1-15. 

NOTE.  Nondeterminism  in  transitions  may  be  taken  care  of  by  partitioning  ?[condition] 
into  those  which  are  controlled  and  uncontrolled  (cf.  [74]).  Disturbances  (and  other 
nondeterminism)  may  be  modeled  by  partitioning  U,V,  and  C  into  portions  that  are  under 
•  the  influence  of  the  controller  or  nature  respectively.  Systems  with  state-output,  edge- 

r.  output,  and  autonomous  and  controlled  jump  delay  maps  (Aa  and  Ac,  respectively)  may 

be  added  as  above.  See  §4.1  for  more  details. 

Our  “unified”  model  for  hybrid  control  is  detailed  in  §5.  Briefly,  it  is  a  controlled  hybrid 
system,  with  the  form 

[z+,  [{lK'‘>feO.«4,{/i)So.u]  ,A,V,G,C,f]  , 

where  each  di  €  Z+.  In  §5  we  show  that  this  model  encompasses  the  discrete  phenomena 
associated  with  Hybrid  systems  [§3.2]  as  well  as  substimes  previously  posed  hybrid  systems 
models  [5,  9,  38,  114,  135,  152,  §§3.3-3.8]. 
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Figure  1-15:  Example  dynamics  of  general  controlled  hybrid  dynamical  system. 


§1.5  MODELING  CONTRIBUTIONS 

REVIEW 

Evidently,  a  hybrid  system  has  continuous  dynamics  modeled  by  a  differential  equation 

x{t)  =  ^(f),  t  >  0 

that  depends  on  some  discrete  phenomena.  Here,  x{t)  is  the  continuous  component  of  the 
state  taking  values  in  some  subset  of  a  Euclidean  spaee.  ^{t)  is  a  controlled  vector  field  that 
generally  depends  on  x{t),  the  continuous  component  u{t)  of  the  control  policy,  and  the 
aforementioned  discrete  phenomena. 

An  examination  of  real-world  examples  and  a  review  of  other  hybrid  systems  models 
has  led  us  to  an  identification  of  these  phenomena.  The  discrete  phenomena  generally 
considered  are  as  follows. 

1.  Autonomous  switching:  Here  the  vector  field  changes  discontinuously  when 
the  state  x(-)  hits  certain  “boundaries.” 

2.  Autonomous  jumps:  Here  the  continuous  state  x(-)  jumps  discontinuously  on  hit¬ 
ting  prescribed  regions  of  the  state  space. 

3.  Controlled  switching:  Here  the  vector  field  ^(0  changes  abruptly  in  response  to  a 
control  command. 
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4.  Controlled  jumps:  Here  the  continuous  state  x(-)  changes  discontinuously  in  re¬ 
sponse  to  a  control  command. 

We  also  review  in  some  detail  the  hybrid  systems  models  of  Witsenhausen  (WHS), 
Tavernini  (TDA),  Back-Guckenheimer-Meyers  (BGM),  Nerode-Kohn  (NKSD),  Antsaklis- 
Stiver-Lemmon  (ASL),  and  Brockett  (BB/BD/BDV).  We  summarize  this  review  as  follows. 

The  WHS,  TDA,  NKSD,  and  ASL  models  combine  ordinary  differential  equations  with 
finite  (or  digital)  automata  by  allowing  the  ODEs  to  depend  on  the  automaton’s  state  or 
output,  while  the  automaton’s  state  or  input  depends  on  a  partitioning  of  the  continuous 
state  space.  The  discrete  state/input  change  on  crossing  these  partition  boundaries.  Each 
model  places  different  restrictions  on  the  allowed  partitioning.  All  these  models  implement 
autonomous  switching. 

The  BGM  model  is  similar,  except  that  it  allows  one  to  make  autonomous  jumps  m  the 
continuous  state,  set  parameters,  or  start  timers  upon  hitting  the  partition  boundaries.  It 
uses  autonomous  switching  and  autonomous  jumps.  ^ 

Brockett  combines  differential  equations  with  a  single  rate  equation,  or  “clock,”  which 
progresses  monotonically.  The  differential  equations  depend  on  the  integer  value  of  this 
clock  variable.  He  also  combines  finite  automata  that  update  on  the  times  when  the  clock 
passes  through  integer  values.  His  BD  model  is  a  special  form  of  autonomous  jumps. 

Comparing  autonomous  versions  of  each  model,  we  see  that  BGM  contains  each  of  the 
others,  while  TDA  is  contained  in  each  of  the  other  autonomous-switching  models.  However, 
BD  and  the  autonomous-switching  models  are  not  strictly  comparable. 

BGM  and  TDA  are  only  autonomous.  From  the  control  perspective,  NKSD  and  ASL 
models  focus  on  the  “control  automaton,”  coding  the  action  of  the  controller  in  the  mappings 
from  continuous  states  to  input  symbols,  through  automaton  to  output  symbols,  and  back 
to  controls.  See  Figure  1-1. 

Witsenhausen  adds  a  control  to  the  continuous  component  of  the  system  dynamics. 
Brockett’s  BD/BDV  models  allow  the  possibility  of  both  continuous  and  discrete  controls  to 
be  exercised  as  input  to  the  continuous  and  symbolic  dynamics  of  the  systems,  respectively. 
These  last  are  closer  to  our  own  study  of  hybrid  control  below.  See  Figure  1-2. 


CLASSIFICATION 

In  this  chapter  we  classify  hybrid  systems  according  to  their  structure  and  the  phenomena 
that  they  exhibit.  The  hierarchy  of  classes  we  explore  are  as  follows  (for  both  autonomous 
and  controlled  systems).  First,  there  are  general  hybrid  dynamical  systems  (GHDS).  These 
are  then  refined  to  the  concept  of  hybrid  dynamical  system,  or  simply  hybrid  system,  studied 
in  this  thesis.  Then  there  are  two  restrictions  of  hybrid  systems,  in  which  the  discrete 
dynamics  are  suppressed,  called  svntched  systems  and  continuous  switched  systems,  which 
are  analyzed  in  §§8-9.  A  further  taxonomy  of  such  systems  in  terms  of  their  structure  and 
the  discrete  phenomena  which  they  admit  is  presented. 

Along  the  way  we  also  discuss  the  dynamics  of  general  hybrid  dynamical  systems.  In 
particular,  we  give  sufficient  conditions  on  GHDS  which  aUow  us  to  construct  its  positive 
orbit,  i.e.,  the  set  of  points  reachable  in  forward  “time  flow.” 

Let  I  =  S\A.  When  a  GHDS  is  time-uniform  (all  semigroups  identical)  with  time¬ 
like  semigroup  F,  it  induces  a  F^-transition  system  [J,F+,$].  In  this  c^e,  we  may  define 
its  (forward)  trajectory  as  a  function  from  F+  into  I.  We  give  sufficient  conditions  for 
existence  and  uniqueness  of  these  trajectories. 
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Finally,  we  also  introduce  switched  systems,  for  example, 

x{t)  =  fi{x{t)),  i  €  Q  ,iV}, 

where  x{t)  €  R"  and  Q  is  called  the  switching  set.  We  add  the  following  switching  rules. 

•  Each  fi  is  globally  Lipschitz  continuous. 

•  The  Fs  are  picked  in  such  a  way  that  there  are  finite  switches  in  finite  time. 

We  also  consider  discrete-time  versions.  Abstracting  away  of  the  finite  dynamics  in  studying 
switched  systems  above  can  be  motivated  by  “verification  by  successive  approximation”  [2]. 
A  continuous  switched  system  is  one  whose  vector  fields  agree  at  switching  times. 

UNIFIED  MODEL 

Finally,  we  come  to  our  unified  model  for  hybrid  control.  We  consider  a  controlled  hybrid 
systems  model,  i.e., 

[Z+,  ,  A,G,F,C,d]  . 

To  ease  presentation,  but  without  real  loss  of  generality,  we  consider  the  continuous  and 
discrete  control  sets  to  be  uniform,  and  that  the  controlled  jump  destinations  are  given  by 
the  sets  £>,  €  D  instead  of  by  the  set- valued  maps  Fi. 

We  also  add  delay  operators  on  autonomous  and  controlled  jumps: 

•  autonomous  jump  delay  Aq  :  A  x  V  — >•  IL).. 

•  controlled  jump  delay  Ac  :  C  x  Dc  R4. . 

The  dynamics  of  the  control  system  is  much  the  same  as  for  GCHDS  above,  except  that 
the  delay  maps  give  rise  to  a  sequence  of  pre-jump  times  {rj}  and  another  sequence  of 
post-jump  times  {F,}  satisfying  0  =  Fo  <  ti  <  Fi  <  T2  <  r2  <  •  •  •  <  00.  On  each 
interval  [Fj_i,Tj)  with  non-empty  interior,  x(-)  evolves  according  to  x{t)  =  fi{x{t),u{t))  in 
some  Xi,  i  €  Z+.  At  the  next  pre-jump  time  (say,  tj)  it  jumps  to  some  £>*  G  Xk  according 
to  one  of  the  following  two  possibilities: 

1.  x{Tj)  6  Ai,  in  which  case  it  must  jump  to  a:(Fj)  =  Gi{x{Tj),Vj)  G  at  time  Fj  = 
Tj  -t-  Aa,i{x{Tj),Vj),  Vj  G  V  being  a  control  input.  We  call  this  phenomenon  an 
autonomous  jump. 

2.  x{Tj)  G  Ci  and  the  controller  chooses  to — it  does  not  have  to— move  the  trajectory 
discontinuously  to  a;(Fj)  E  D  at  time  Fj  =  Tj  -1-  Ac,i(x(Tj),a:(Fj)).  We  call  this  a 
controlled  (or  impulsive)  jump. 

See  Figure  1-15. 

Thus,  the  admissible  control  actions  available  are 

•  the  continous  controls  u(-),  exercised  in  each  constituent  regime, 

•  the  discrete  controls  {ui}  exercised  at  the  pre-jump  times  of  autonomous  jumps 
(which  occur  on  hitting  the  set  A), 
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•  the  pre-jump  or  intervention  times  and  associated  destinations  {x(C0}  of 
the  controlled  jumps. 

We  then  explicitly  show  that  the  above  model  captures  all  identified  discrete  phenomena 
arising  in  hybrid  systems  and  subsumes  all  reviewed  and  classified  hybrid  systems  models. 
The  resulting  model  is  useful  for  posing  and  solving  hybrid  control  problems  in  the  sequel. 

§1.6  ANALYSIS  CONTRIBUTIONS 

TOPOLOGICAL  RESULTS 

In  traditional  feedback  control  systems — continuous-time,  discrete-time,  sampled-data  the 
maps  from  output  measurements  to  control  inputs  are  continuous  (in  the  usual  metric-based 
topologies).  Continuity  of  state  evolution  and  controls  with  respect  to  the  states  also  plays 
a  role.  Yet,  in  general,  hybrid  systems  are  not  continuous  in  the  initial  condition: 

Example  1.2  Consider  the  following  hybrid  system  on  Xi  =  X2  =  The  continuous 
dynamics  is  given  by  fi  =  (1,  0)^  and  f2  =  (0,  1)^.  The  discrete  dynamics  is  given  by 
Ai  =  [0,  if  and  G{x,  1)  =  (a:,2).  Now  consider  the  initial  conditions  x(0)  =  (-e,  -e)^  and 
y(0)  =  (-e,0)^.  Note  that  a;(l)  =  (1  -  e,  -e)  but  y{l)  =  (0, 1  -  e).  Clearly,  no  matter  how 
small  e,  hence  llx(O)  -  y(0)lloo,  is  chosen,  ||x(l)  -  y(l)||oo  =  1- 

Wie  examine  systems  as  in  Figure  1-1,  where  the  set  of  symbols,  automaton  states,  and 
outputs,  are  finite  sets  and  the  plant  and  controls  belong  to  a  continuum.  However,  note 
that  the  only  continuous  maps  from  a  connected  set  to  a  disconnected  one  are  the  constant 
ones.  Hence,  the  usual  discrete  topologies  on  a  set  of  symbols  do  not  lead  to  nontrivial 
continuous  AD  maps. 

We  then  examine  topologies  that  lead  to  continuity  of  each  member  of  topologies  for 
which  the  AD  maps  from  measurements  to  symbols  are  continuous.  In  dynamics  terms,  we 
examine  topologies  that  lead  to  continuity  of  each  member  of  the  family  of  maps  G  o  (pg. 
One  topology  in  particular,  proposed  by  Nerode  and  Kohn  [114],  is  studied  in  depth. 

We  then  look  at  what  happens  if  we  attempt  to  “complete  the  loop”  in  Figure  1-1,  by 
also  considering  the  DA  maps.  We  exhibit  a  topology  making  the  whole  loop  continuous. 
But,  instead  of  dwelling  on  this,  we  examine  a  different  view  of  hybrid  systems  as  a  set  of 
continuous  controllers,  with  switching  among  them  governed  by  the  discrete  state.  Also, 
we  examine  fuzzy  control  systems  consisting  of  a  finite  set  of  so-called  fuzzy  rules.  On  the 
surface,  they  are  hybrid.  Yet,  we  show  that  fuzzy  control  leads  to  continuous  maps  (firom 
measurements  to  controls)  and  that  all  such  continuous  maps  may  be  implemented  via  fuzzy 
control. 

COMPLEXITY  RESULTS 

Computational  equivalence  (or  simulation  of  computational  capabilities)  may  be  shown  in 
the  following  two  ways: 

1.  Comparing  accepted  languages  [76], 

2.  Simulation  of  step-by-step  behavior  [20]. 

The  above  are  clear  when  comparing  two  digital  automata.  The  situation  is  slightly  harder 
in  comparing  digital  and  analog  systems. 
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In  order  to  examine  the  computational  capabilities  of  hybrid  and  continuous  systems 
we  first  must  introduce  notions  of  a  continuous-time  system  simulating  a  discrete-time  one. 
[31,  §7.2]: 

Definition  1.3  A  continuous-time  transition  system  [X,]R+,/]  simulates  via  section  or 
S-simulates  a  discrete-time  transition  system  [Y,  Z+,  F]  if  there  exist  a  continuous  sur¬ 
jective  partial  function  x/}  :  X  Y  and  to  €  K+  such  that  for  all  x  ^  xp 
k  € 

xp{f{x,kto))  =  F{xp{x),k). 

Definition  1.4  A  continuous-time  transition  system  [X,  1R+,/]  simulates  via  intervals 
or  I-simulates  a  discrete-time  transition  system  [y,  Z+,  F]  if  there  exist  a  continuous  sur¬ 
jective  partial  function  xp  :  X  —¥  Y  and  e  >  0  such  that  V  =  xp  ^(1^)  is  open  and 
for  all  X  e  V  the  set  T  =  {t  €  R+  \  f{x,t)  e  V}  is  a  union  of  intervals  {Tk,Tjf), 
0  =  To  <  <  n  <  <  •  •  •,  -  Tfcl  >  e,  xvith 

xp{f{x,tk))  =  F{xp{x),k), 


for  all  tk  €  {Tk,Tk)- 

When  the  continuous-time  transition  system  is  a  HDS,  the  maps  xp  above  can  be  viewed  as 
an  edge-output  and  state-output  map,  respectively.  In  this  case,  S-simulation  can  be  viewed 
as  equivalent  behavior  at  equally-spaced  edges  and  I-simulation  as  equivalent  behavior  in 
designated  nodes.  S-  and  I-simulation  are  distinct  notions.  Sl-simulation  denotes  the  case 
when  both  hold. 

In  §7  we  show  the  following: 

•  Every  dynamical  system  [R",Z,F]  can  be  S-simulated  by  an  autonomous-switching, 
two-discrete-state  hybrid  system  on  . 

•  Every  dynamical  system  [R” ,  Z+,  F]  can  be  S-simulated  by  an  autonomous-jump,  two- 
discrete-state  hybrid  system  on  R” . 

•  Every  dynamical  system  [Y,Z,F],  Y  C  Z”,  can  be  Sl-simulated  by  a  (continuous) 
dynamical  system  of  the  form  [R2"+i  ,  R+  ,  /].  Furthermore,  if  Y  is  bounded  /  can  be 
taken  Lipschitz  continuous. 

As  corollaries  to  the  last  we  result,  we  have  (via  demonstrated  isomorphisms  with  dynamical 
systems  on  Z) 

•  Every  Turing  machine,  pushdown  automata,  and  finite  automaton  can  be  Sl-simulated 
by  a  (continuous)  dynamical  system  of  the  form  [IE^,R+,/]. 

•  Using  Sl-simulation,  there  is  a  system  of  continuous  DDEs  in  R^  with  the  power  of 
universal  computation. 

Noting  that  even  ordinary  dynamical  systems  are  so  computationally  powerful,  we  use 
the  the  famous  asynchronous  arbiter  problem  [26, 100,  144]  to  distinguish  between  dynam¬ 
ical  and  hybrid  systems. 

NOTE.  An  arbiter  is  a  device  that  can  be  used  to  decide  the  winner  of  two-person  races 
(within  some  tolerances).  It  has  two  input  buttons,  Bi  and  B2,  and  two  output  lines,  Wi 
and  W2,  that  can  each  be  either  0  or  1.  For  its  “technical  specifications”  see  §7.4.1. 
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In  pairticular,  we  settle  the  problem  in  an  ODE  framework  by  showing  that  no  system 
of  the  form 

with  /  Lipschitz  and  h  continuous,  can  implement  an  arbiter  [26,  31].  On  the  other  hand, 
we  exhibit  a  hybrid  system  of  the  form 

[{1, 2},  [r,lR+ , B,  {/i, /2}, /i] ,  A, G] , 

with  each  fq  Lipschitz,  h  continuous,  and  G  autonomous-switching,  that  satisfies  the  arbiter 
specifications. 


ANALYSIS  TOOLS 

In  the  first  part  of  §8,  we  develop  general  tools  for  analyzing  continuous  switching  sys¬ 
tems.  For  instance,  we  prove  an  extension  of  Bendixson’s  Theorem  to  the  case  of  Lipschitz 
continuous  vector  fields.  This  gives  us  a  tool  for  analyzing  the  existence  of  limit  cycles  of 
continuous  switching  systems.  We  also  prove  a  lemma  dealing  with  the  continuity  of  differ¬ 
ential  equations  with  respect  to  perturbations  that  preserve  a  linear  part.  Colloquially,  this 
lemma  demonstrates  the  robustness  of  ODEs  with  a  linear  part.  For  purpose  of  discussion, 
we  call  it  the  Linear  Robustness  Lemma.  This  lemma  is  useful  in  easily  deriving  some  of 
the  common  robustness  results  of  nonlinear  ODE  theory  (as  given  in,  for  instance,  [11]). 
This  lemma  also  becomes  useful  in  studying  singular  pertmbations  if  the  fast  dynamics  are 
such  that  they  maintain  the  corresponding  algebraic  equation  to  within  a  small  deviation. 
We  add  some  simple  propositions  that  allow  us  to  do  this  type  of  analysis  in  §9. 

In  the  second  part  of  §8,  we  examine  stability  of  we  introduce  “multiple  Lyapunov 
functions”  as  a  tool  for  analyzing  Lyapunov  stability  of  switched  systems.  The  idea  here  is  to 
impose  conditions  on  switching  that  guarantee  stability  when  we  have  Lyapunov  functions 
for  each  system  fi  individually.  Iterative  function  systems  are  presented  as  a  tool  for 
proving  Lagrange  stability  and  positive  invariance.  We  also  address  the  case  where  the 
finite  switching  set  is  replaced  by  an  arbitrary  compact  set. 


ANALYZING  EXAMPLES 

We  have  analyzed  example  systems  arising  fi:om  a  realistic  aircraft  controller  problem  which 
logically  switches  between  two  controllers  (one  for  tracking  and  one  for  regulation  about  a 
fixed  angle  of  attack)  in  order  to  achieve  reasonable  performance  and  safety.  While  stability 
of  such  hybrid  systems  has  previously  only  been  examined  using  simulation  [134],  we  were 
able  to  prove  global  asymptotic  stability  for  a  meaningful  class  of  cases  [28].  Using  our 
robustness  lemma  to  compare  ODE  solutions,  we  extend  the  result  to  a  class  of  continuations 
of  these  systems,  in  which  dynamically  smooths  the  logical  nonlinearity. 

The  conclusion  in  this  case  is  that  the  continuation  method  worked  in  reverse,  i.e.,  it  was 
easier  to  prove  stability  of  the  original,  hybrid  system  directly.  Furthermore,  we  concluded 
stability  of  the  continuation  via  that  of  the  original  system.  In  effect,  we  showed  robustness 
of  the  max  system  to  the  considered  class  of  dynamic  continuations. 
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§1.7  CONTROL  CONTRIBUTIONS 

THEORETICAL  RESULTS 

We  consider  the  following  optimal  control  problem  on  our  unified  hybrid  systems  model. 
Let  a  >  0  be  a  discount  factor.  We  add  to  our  model  the  following  known  maps; 


•  Running  cost  k  :  S  xU  -4  IR+ . 

•  Autonomous  jump  cost  or  transition  cost  Ca-  Ax  V  -4  1+. 

•  Controlled  jump  cost  or  impulse  cost  Cc  :  C  x  Dc  -4  K+. 


The  total  discounted  cost  is  defined  as 

/  e"“‘fe(a;(t),u(t))  dt  +  e"“^'cc(a;(Ci),x(Ci))  (1-2) 

Jt  i  i 

where  T  =  ]R+\(Uih,ri)),  (respectively  {Ci})  are  the  successive  pre-jump  times  for 
autonomous  (respectively  impulsive)  jumps  and  is  the  post-jump  time  for  the  jth  impul¬ 
sive  jump.  The  decision  or  control  variables  over  which  Equation  (1.2)  is  to  be  minimized 
are  the  admissible  controls  of  our  unified  model. 

Under  some  assumptions  (the  necessity  of  which  are  shown  via  examples)  we  have  the 

following  results  [32]: 

•  A  finite  optimal  cost  exists  for  any  initial  condition.  Furthermore,  there  axe  only 
finitely  many  autonomous  jumps  in  finite  time. 

•  Using  the  relaxed  control  framework,  an  optimal  trajectory  exists  for  any  initial  con¬ 
dition. 

•  For  every  e  >  0  an  e-optimal  control  policy  exists  wherein  u{-)  is  precise,  i.e.,  a  Dirac 
measure. 


•  The  value  function,  V,  associated  with  the  optimal  control  problem  is  continuous  on 
S\{dAUdC)  and  satisfies  the  following  generalized  quasi-variational  inequalities 
(GQVIs). 


1.  a:  €  S\A: 


2.  On  C: 


3.  On  A: 


F{x,  u)  =  (ViF (x),  /i(x,  u))  -  aV (x)  +  k{x,  u), 
min  F(x,u)  <  0. 


V’(x)  <  min  { Cc(x,  2)  -I-  e  (2)} 

z^D  ^  ^ 

V{x)  <  mm{ca(x,t;)  -H  e-“^«(*-'')U(G(x,i;))} 


4.  On  C: 


(1) .  (2)  =  0 
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ALGORITHMS  AND  EXAMPLES 

In  this  section  we  outline  four  approaches  to  solving  the  generalized  quasi-variational  in- 
equalities  (GQVIs)  associated  with  optimal  hybrid  control  problems.  Our  algorithmic  basis 
for  solving  these  GQVIs  is  the  Bellman  Equation; 

V*{x)  =  min{p(x,a)  +  V*(x'(x,  a))}  , 
p^n 

where  n  is  a  generalized  set  of  actions.  The  three  classes  of  actions  available  in  our  hybrid 
systems  framework  at  each  x  axe 

•  Continuous  Controls:  u  £U. 

•  Controlled  Jumps:  choosing  source  and  destination  (if  x  G  C). 

•  Autonomous  Jumps:  possibly  modulated  by  discrete  controls  u  €  V  (if  x  €  A). 

Prom  this  viewpoint,  generalized  policy  and  value  iteration  become  solution  tools. 

The  key  to  efficient  algorithms  for  solving  optimal  control  problems  for  hybrid  sys¬ 
tems  lies  in  noticing  their  strong  connection  to  the  models  of  impulse  control  [§2.2.4]  and 
piecewise-deterministic  processes  [§2.2.5].  Making  this  explicit,  we  develop  algorithms  sim- 
ilar  to  those  for  impulse  control  and  one  based  on  linear  programming. 

Three  illustrative  examples  are  solved.  They  are  as  follows.  First,  we  consider  a  hys¬ 
teresis  system  that  exhibits  autonomous  switching  and  has  a  continuous  control.  Then  we 
discuss  a  satellite  station-keeping  problem.  The  on-off  nature  of  the  satellite’s  reaction  jets 
creates  a  system  involving  controlled  switching.  We  end  with  a  transmission  problem.  The 
goal  is  to  find  the  hybrid  strategy  of  continuous  accelerator  input  and  discrete  gear-shift 
position  to  achieve  maximum  acceleration.  In  each  case,  the  optimal  controls  produced 
verify  engineering  intuition. 


Chapter  2 

Preliminaries  and  Related  Work 


In  this  chapter,  we  cover  preliminaries  and  review  some  literatmre  related  to  hybrid  systems. 

§2.1  PRELIMINARIES 

Throughout  the  thesis  we  assume  familiarity  with  the  notation  and  concepts  of  analysis 
[126],  topology  [62,  75,  113,  §A],  ordinary  differential  equations  [6,  73],  automata  theory 
[20,  76],  control  theory  [58,  85,  96,  133],  and  nonlinear  systems  analysis  [132,  142]. 

Next,  we  collect  some  notation  used  throughout.  Then  we  cover  the  necessary  prelim¬ 
inary  information  from  dynamical  systems,  ordinary  differential  equations,  and  automata 
theory.  Some  review  of  topology  is  done  in  §A.  A  majority  of  the  notation  is  collected 
in  the  Symbol  Index,  pp.  193-194.  Finally,  a  general  Index  is  provided  for  convenience  in 
locating  definitions. 

§2.1.1  NOTATION 

New  concepts  being  defined  are  displayed  in  bold  face.  The  end  of  a  proof  is  denoted  with 
the  symbol  □.  Sectional  cross-referencing  within  the  thesis  is  done  using  the  symbol  §. 

A  system  is  the  abstract  entity  of  our  study  that  we  shall  not  formally  define,  just  as 
“points”  are  not  defined  in  analysis.  Difference  and  differential  equations  are  examples  of 
frequently  used  systems.  A  map  is  a  function;  we  use  the  two  interchangeably. 

We  make  use  of  common  abbreviations  like  ODEs  (ordinary  differential  equations),  FA 
(finite  automata/on),  DEDS  (discrete  event  dynamical  systems;  see  [74]),  etc. 

The  symbols  M,  1+,  Z,  and  Z+  denote  the  reals,  nonnegative  reals,  integers,  and  non¬ 
negative  integers,  respectively.  For  a:  €  E,  [xj  denotes  the  greatest  integer  less  than  or  equal 
to  X,  and,  in  an  abuse  of  common  notation,  fx]  denotes  the  least  integer  greater  than  x.  |A|, 
A  a  set,  denotes  its  cardinality;  A  —  B,  when  A  and  B  are  sets,  is  read  A  is  isomorphic 
to  B  and  means  herein  that  |A|  =  |B|;  iV  denotes  the  set  {1, 2, ... ,  N}]  =  means  “defined 
equal  to,”  whereas  :=  denotes  reassignment  of  meaning. 

Below  we  deal  with  continuous  time  systems,  such  as  ODEs,  that  are  affected  by  events 
at  discrete  instants,  such  as  jumps  in  state.  We  use  [t]  to  denote  the  time  less  than  or  equal 
to  t  at  which  the  last  “jump”  or  “event”  occurred.  If  the  event  is  unclear,  we  subscript  the 
variable,  so  that  [t]p  denotes  the  time  at  which  the  variable  p  last  jumped.  Throughout, 
v[p\  and  x[t]  are  shorthand  for  i;([pj)  and  x([t]),  respectively  (cf.  [38]).  Finally,  we  use 
Sontag’s  discrete-time  shift  operator  [133]  to  denote  the  discrete  phenomena  updating  a 
hybrid  systems’  states.  That  is,  is  the  successor  of  q.  Likewise,  q~  is  its  predecessor. 

Other  notation  is  common  [113,  126].  For  example,  X\U  represents  the  complement 
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of  U  in  X-,U  represents  the  closure  of  U,  U°  its  interior,  dU  its  boundary; 
denote  the  right-hand  and  left-hand  limits  of  the  function  /  at  t,  respectively;  a  function  is 
right-continuous  if  /(t+)  =  f{t)  for  all  t;  C{X,Y)  denotes  the  space  of  continuous  functions 
with  domain  X  and  range  Y',  denotes  the  transpose  of  vector  u;  unless  specified  ||a:|| 
denotes  an  arbitrary  norm  of  vector  r,  ||aj|l2  its  Euclidean  norm;  the  infinity  norm  of  x  €  M  , 
denoted  ||x||oo,  is  maxf,^!  |xi|. 

§2.1.2  DYNAMICAL  SYSTEMS 

First,  we  review  some  standard  definitions  fi:om  dynamical  systems  [67, 129].  A  dynamical 
system  is  a  system  S  =  [X,r,  </>],  where 

•  X  is  an  arbitrary  topological  space,  the  state  space  of  E. 

•  The  transition  semigroup  F  is  a  topological  semigroup.^ 

•  The  (extended)  transition  map  ^  :  X  x  F  — >  X  satisfies  the  following. 

1.  Identity.  For  all  x  G  X,  <f>{x,0)  =  x. 

2.  Semigroup.  For  all  x  €  X  and  arbitrary  gi,  §2  in  F, 

91)^92)  =  4>{x,9\  +52)- 

3.  Continuity,  (j)  is  continuous  in  both  arguments  simultaneously,  i.e.,  for  any 
neighborhood  W  of  the  point  (t>{x,g)  there  exist  neighborhoods  U  and  V  of  the 
point  X  and  the  element  g  respectively  such  that  <p{U,  V)  CW. 

Such  objects  are  well-studied  in  mathematics  under  the  names  topological  transformation 
groups  or  continuous  general  dynamical  system.  A  general  dynamical  system  need  not 
have  F  a  topological  space  and  only  requires  continuity  with  respect  to  x  [129]. 

Technically,  we  have  defined  semi-dynamical  systems,  with  the  term  dynamical  sys¬ 
tem  reserved  for  the  case  F  above  is  a  group.  However,  the  more  “popular”  notion  of  dynam¬ 
ical  system  in  math  and  engineering— and  the  one  used  here— requires  only  the  semigroup 
property  [59,  96].  Thus,  the  term  reversible  dynamical  system  is  used  when  it  is  necessary 

to  distinguish  the  group  from  semigroup  case  [97]. 

If  a  dynamical  system  is  defined  on  a  subset  of  X,  we  say  it  is  a  dynamical  system  in 
X.  For  every  fixed  value  of  the  parameter  g,  the  function  (t>{-,g)  defines  a  mapping  of  the 
space  X  into  itself.  Given  [X,Z+,<^],  <f>i;  1)  is  its  transition  function.  Thus  if  [X,Z+,4>] 
is  reversible,  its  transition  function  is  invertible,  with  inverse  given  by  ^(•,  -1). 
Generalizing  this  notion,  we  also  denote  by  dynamical  system?'  a  system 

E  =  [X,F,/], 

where  X  and  F  axe  as  above,  but  the  transition  function  /  is  the  generator  of  the 
extended  transition  function  <l>.  Thus  /  :  X  x  Fj  ->  X,  where  Tg  generates  F.  For  example, 

^  We  assume  only  semigroups  with  identity,  (a.k.a.  monoids).  We  use  addition  notation  for  the  semigroup 
operation  throughout  (i.e.,  -1-  is  the  operation,  0  the  identity)  and  suppress  the  operation  in  referencing  the 

semigroup. 

^The  distinction  should  be  clear  by  context. 
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in  the  case  of  r  =  Z,  /  :  X  X  is  given  by  /  =  (f>{-,  1).  In  the  case  of  T  =  M,  /  :  X  ->  TX 
is  given  by 

f{x)  =  Y  4>{x,  t) 

at  t=o 

This  definition  is  just  as  general  as  before  since  /  may  depend  on  all  of  F.  Of  particular 
interest  are  cases  where  F  is  finitely  generated.  For  example,  we  had  it  depend  on  1  (the 
generator  for  Z)  and  dt  (informally,  the  generator  for  R)  in  the  examples  above.  Another 
concrete  example  is  a  finite  automaton  given  by  its  transition  function,  u:  Qx  I  -^Q  (since 
I  generates  the  free  monoid  I*  on  which  its  extended  transition  function  is  defined)  [20]. 

Examples  of  dynamical  systems  abound,  including  autonomous  ODEs,  autonomous  dif¬ 
ference  equations,  finite  automata,  pushdown  automata,  Turing  machines,  Petri  nets,  etc. 

The  set  (p{x,  F)  =  {^(a:,  g)  :  g  E  T}  is  called  the  orbit  of  the  point  x.  The  function 
F  <f){x,  F)  trajectory  of  the  point  x.  An  equilibrium  or  fixed  point  of  [X,  T,<j>]  is  a 
point  X  such  that  (p{x,  g)  =  x  for  all  5  €  F.  A  set  A  C  X  is  invariant  with  respect  to  (f>,  if 
4>{A,g)  C  A  for  all  g  ET. 

The  notions  of  equivalence  and  homomorphism  are  crucial.  Two  dynamical  systems 
[X,F,0],  [y,F,  0]  are  said  to  be  isomorphic  or  (topologically)  equivalent  if  there  exists 
a  homeomorphism  ip  :  X  -^Y  such  that 

ip{(p{x,g))  =  d{ip{x),g), 

for  all  X  E  X  and  g  E  T.  If  the  mapping  ip  is  only  continuous,  then  [X,F,</)]  is  said  to 
be  homomorphic  to  [y,F,  0].  Homomorphisms  preserve  trajectories,  fixed  points,  and 
invariant  sets. 

Less  restrictively,  a  transition  system  is  a  general  dynamical  system  as  above,  except 
that  (p  need  not  be  continuous  in  x.  As  shorthand  we  use  T-transition  system  to  denote 
one  with  group  F.  Continuous-  and  discrete-time  transition  system  denote  the  cases 
where  F  =  R  (or  R+)  and  F  =  Z  (or  Z+),  respectively. 

Next,  we  refine  the  above  concept  by  introducing  controlled  dynamical  systems  and 
systems  with  marked  states,  output,  and  timing.  In  each  case,  X,  F,  <^,  and  /  are  as  above. 

A  controlled  dynamical  system  or  a  dynamical  system  with  input  is  a  system 

S  =  [X,  F,  U,  /],  (or  S  =  [X,  F,  U,  <(>]) , 

where  U  is  the  set  of  inputs.  The  input  ueU  may  act  state-by-state,  i.e.  f  :  X  xTgXU  ^ 
X,  or  by  transition,  i.e.  <p:XxTxU-^X. 

To  each  of  the  above,  we  may  append  the  following  refinements: 

•  Output.  Add  y  the  set  of  outputs.  The  output  map  h  may  produce  state-output, 
i.e.  h  ;  X  y,  or  edge-output,  i.e.  h:  X  xT  -^Y. 

NOTE.  These  are  often  equivalent.  For  example,  Moore  (resp.  Mealy)  machines  are 
finite  automata  with  state-output  (resp.  edge-output).  However,  Moore  and  Mealy 
machines  are  equivalent  [76].  In  differential  equations  one  almost  always  assumes 
state-output,  the  edge-output  case  being  subsumed  in  most  cases  of  interest  by  inte¬ 
gration  operators.  There  is  also  the  equivalence  of  nodes  and  actions  under  perfect 
recall  in  game  theory  [115]. 
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•  Marked  states.  These  are  distinguished  subsets  of  X.  For  example,  let  /  C  X 
denote  a  set  of  (admissible)  initial  or  start  states,  and  F  C  S  the  set  of  final, 
halting,  or  cemetery  states.  Roughly,  such  a  dynamical  system,  is  defined  to  start 
from  any  point  I,  from  which  it  evolves  until  it  hits  a  point  in  F,  at  which  time  its 
action  is  halted. 

•  Timing.  Add  a  transition  time  map  or  timing  map.  r  ;  X  x  F  ->  R  (or  I4). 
Of  particular  interest  are  maps  where  t  is  constant  on  X  and  those  that,  in  addition, 
are  homomorphisms  with  subsets  of  R  (R+).  In  the  generator  case,  the  timing  map 
can  be  defined  on  X  x  Fg. 

§2.1.3  ODEs 

In  this  thesis,  the  continuous  dynamical  systems  dealt  with  are  defined  by  the  solutions  of 
ordinary  differential  equations  (ODEs)  [73]: 

i;(t)  = /(x(t)),  (2.1) 

where  x{t)  e  X  C  R”.  The  function  /  :  X  R”  is  called  a  vector  field  on  R”.  The 
resulting  dynamical  system  is  then  given  by  (t>{xo,t)  =  x{t)  where  x(-)  is  the  solution  to 
Equation  (2.1)  starting  at  xo  at  t  =  0.  We  assume  existence  and  uniqueness  of  solutions;  see 
[73]  for  conditions.  A  well-known  sufficient  condition  is  that  the  vector  field  /  is  Lipschitz 
continuous.  That  is,  there  exists  L  >  0  (called  the  Lipschitz  constant)  such  that 

\\f{x)- f{y)\\<L\\x-y\\,  forallx,yeX. 

A  system  of  ODEs  is  called  autonomous  or  time-invariant  if  its  vector  field  does 
not  depend  explicitly  on  time.  Throughout,  the  shorthand  continuous  (resp.  Lipschitz) 
ODEs  to  denote  those  with  continuous  (resp.  Lipschitz)  vector  fields. 

An  ODE  with  inputs  and  outputs  [96,  133]  or  plant  is  given  by 

xit)  —  (2.2) 

y{t)  =  h{x{t)). 

Where  x{t)  G  X  C  R",  u{t)  €  C/  C  R”*,  y  €  F  C  RP,  /  :  R"  x  R'"  R",  and  h-.W  ^W. 

The  functions  ii(')  and  j/(‘)  are  the  inputs  and  outputs,  respectively. 

Whenever  inputs  are  present,  as  in  Equation  (2.2),  we  say  /  is  a  controlled  vector 
field. 

§2.1.4  DIGITAL  AUTOMATA 

We  begin  with  some  standard  material  [76].  A  symbol  is  the  abstract  entity  of  automata 
theory.  Examples  are  letters  and  digits.  An  alphabet  is  a  finite  set  of  symbols.  A  string 
is  a  finite  sequence  of  juxtaposed  symbols.  The  empty  string,  denoted  e,  is  the  string 
consisting  of  zero  symbols. 

A  finite  automaton  (FA)  is  a  system  (Q,  /,  u,  qo,  M),  where 
•  Q  is  a  finite  set  of  states, 


•  J  is  an  alphabet,  called  the  input  alphabet. 


§2.1  Preliminaries  43 


•  u  is  the  transition  function  mapping  Q  x  I  into  Q, 

•  go  £  Q  is  the  initial  state, 

•  M  C  Q  is  the  set  of  the  accepting  states. 

We  envision  a  finite  automaton  as  a  finite  control  reading  a  sequence  of  symbols  from  I 
written  on  a  tape.  In  one  move,  the  finite  automaton  in  state  q  scans  symbol  a,  enters  state 
i/(g,  a),  and  moves  its  head  one  symbol  to  the  right.  If  the  head  moves  off  the  right  end  of 
the  tape  while  in  an  accepting  state,  then  it  accepts  the  entire  tape. 

A  (deterministic)  pushdown  automaton  (in  normal  form),  or  simply  PDA,  is  a 
system  M  =  {Q,r,I,i',qo,Zo,F),  where 

•  Q  is  the  finite  set  of  states, 

•  r  is  an  alphabet,  called  the  tape  alphabet, 

•  /  is  an  alphabet,  called  the  input  alphabet, 

•  1/  is  the  transition  function  firom  Q  xT  x  {lU  {e})  to  Q  x  (F  U  {e}), 

•  go  in  <5  is  the  start  state, 

•  Zo  in  r  is  the  start  symbol, 

•  F  C  Q  is  the  set  of  final  states. 

Thus,  we  envision  a  finite  control  reading  a  sequence  of  symbols  of  I  written  on  a  tape  and 
manipulating  a  stack  of  symbols.  In  state  q  and  with  Z  the  topmost  stack  symbol,  then 
exactly  one  of  the  following  is  true.^ 

1.  t/{q,  Z,i)  =  {p,Y)  for  some  i  €  /, 

2.  iy{q,Z,e)  =  (p,Y). 

If  y  €  r,  the  PDA  performs  a  push  operation,  i.e.,  it  replaces  Z  with  ZY ,  and  Y  becomes 
the  new  top  stack  symbol.  If  y  =  e,  it  performs  a  pop,  i.e.,  it  removes  Z  from  the  stack. 
In  the  first  case,  the  PDA  performs  a  push  or  pop  (depending  on  y),  moves  to  state  p,  and 
advances  the  input  head.  In  the  second  case,  the  PDA  performs  a  push  or  pop  (depending 
on  y)  and  moves  to  state  p  (i.e.,  it  ignores  the  tape  input  and  does  not  advances  the  input 
head).  The  PDA  is  assumed  to  halt  on  an  empty  stack  or  the  state’s  entering  F. 

In  the  1930’s  Alan  Turing  introduced  a  mechanical  model  for  computation  which  con¬ 
sisted  of  a  finite  control,  an  (infinite  to  the  right)  input  tape  that  is  divided  into  cells,  and  a 
tape  head  that  scans  one  cell  of  the  tape  at  a  time.  Each  tape  cell  may  hold  one  of  a  finite 
number  of  tape  symbols.  Initially,  the  machine  starts  with  its  tape  head  at  the  leftmost 
cell,  its  internal  control  at  some  designated  initial  state,  and  its  input  coded  into  the  first 
0  <  n  <  oo  tape  cells.  All  remaining  cells  hold  a  special  symbol,  called  the  blank,  which  is 
separate  from  the  input  symbols.  More  formally  a  Turing  machine  (TM)  is  a  system 

M=  {Q,T,BJ,i/,qo,F) , 


where 


®By  determinism.  Also  we  may  assume  1/  is  a  total  function  by  appending  special  final  states. 
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•  Q  is  the  finite  set  of  states, 

•  r  an  alphabet  of  allowable  tape  symbols, 

•  B,  a  symbol  of  T,  is  the  blank, 

•  /,  a  subset  of  F  not  including  B,  is  the  alphabet  of  input  symbols, 

•  1/  is  the  next  move  function,  a  mapping  from  QxFtoQxFx  {L,  R} , 

•  qoinQ  is  the  start  state, 

•  F  C  Q  is  the  set  of  final  states. 

Evidently,  a  Turing  machine  is  a  finite  automaton  whose  input  at  step  k  is  the  tape  symbol 
in  the  cell  over  which  the  tape  head  is  positioned  at  the  fcth  step.  In  addition  to  changing 
state,  however,  the  Turing  machine  can  also  print  a  symbol  on  the  scanned  tape  cell  and 
move  its  head  left  or  right  one  cell. 

Now  we  move  on  to  some  particular  notation.  An  inputless  FA  (resp.  PDA)  is  one 
whose  input  alphabet  is  empty,  i.e.,  one  whose  transition  function  depends  solely  on  its 
state  (resp.  state  and  top  stack  symbol). 

A  digital  or  symbolic  automaton  (with  input  and  output)  is  a  system  (Q,  I,  t',  O,  jj), 
consisting  of  the  state  space,  input  alphabet,  transition  function,  output  alphabet,  and 
output  function,  respectively.  We  assume  that  Q,  /,  and  O  are  each  isomorphic  to  subsets 
of  Z+.  When  these  sets  are  finite,  the  result  is  a  finite  automaton  with  output.  In  any 
case,  the  functions  involved  are  u  :  Q  x  I  Q  and  rj :  Q  x  I  O.  The  “dynamics  of  the 
automaton  are  given  by 

Qk+i  = 

Ok  =  »?(9fe»*fe)- 

Such  a  model  is  easily  seen  to  encompass  finite  automata.  Mealy  and  Moore  machines, 
pushdown  automata,  Turing  machines,  Petri  nets,  etc. 


§2.2  RELATED  WORK 

Here,  we  briefly  mention  other  areas  of  inquiry  related  to  hybrid  systems.  The  interested 
reader  is  referred  to  the  cited  works  for  more  details.  Some  connections,  e.g.,  impulse  control 
to  hybrid  systems  control,  are  made  clear  later  in  the  thesis. 

§2.2.1  VARIABLE  STRUCTURE  AND  SWITCHED  SYSTEMS 

Switched  systems  have  been  looked  at  by  [30,  93,  119,  149]  and  others.  We  discuss  them  in 
§4.5  and  §8.  In  variable  structure  systems  [139],  one  takes  a  plant  described  by 

X  =  /(x,  it,t). 


Each  component  of  the  control  is  assumed  to  undergo  discontinuity  on  an  appropriate 
surface  in  the  state  space: 


uf{x,t), 

(x,t). 


if  Si{x)  >  0, 
if  Si{x)  <  0. 
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The  design  problem  is  to  choose  the  continuous  functions  uf ,  uj' ,  and  the  functions  Si{x) 
so  that  the  system  behaves  in  a  desired  manner  (e.g.,  globally  stabilizization  about  some 
operating  point). 

§2.2.2  JUMP  SYSTEMS 

Also  related  to  hybrid  systems  are  so-called  jump  systems.  The  majority  of  results  are 
for  continuous-time  linear  systems  with  Markovian  jumps.  See  [82,  101]  and  the  references 
therein.  These  systems  are  modeled  by 

x{t)  =  A{t,  r{t))  x{t)  -I-  B{t,  r{t))  u{t), 

where  t  G  [to,  T],  T  may  be  finite  or  infinite,  x{t)  €  M”  (t)  is  the  ®-process  state,  u{t)  G  R"" 
is  the  ®-process  input,  and  A{t,r{t))  and  B{t,r{t))  are  appropriately  dimensioned  real¬ 
valued  matrices,  which  are  functions  of  the  random  process  {r(t)}.  The  form  process 
{r{t)}  is  a  continuous-time  discrete-state  Markov  process  taking  values  in  N  with  transition 
probability  matrix  P  =  {pij}  given  by 

Pij  =  Pr{r(t  +  A)=j\  r{t)  =  f}, 

^  f  AyA-f-O(A),  iff#i, 

1  -h  AiiA  -h  0(A),  if  i  =  j, 

where  A  >  0,  Ay  >  0  is  the  form  transition  rate  from  i  to  j  {i  ^  j),  and 

N 

Xii  =  —  Ay. 

Note  that,  in  general.  A,  B,  and  the  Ay’s  could  be  explicit  functions  of  time.  Also,  the 
switchings  are  nondeterministic  and  do  not  depend  on  the  continuous  state. 

In  [60]  a  special  class  of  hybrid  systems  is  studied  in  which  the  Markov  process  above  is 
replaced  with  a  deterministic  one: 

x{t)  =  A{r{t))x{t)  + B{r{t))u{t), 
y{t)  =  0(r(<))x(<), 

with  r{t)  e  N.  Stability,  controllability,  and  observability  results  are  given  for  the  two  cases 
described  below.  Let  Sn  denote  the  set  of  permutations  of  the  values  in  iV  and  niunbers 
6ti  >  0  be  given.  The  convention  is  that  the  ith  system  is  active  for  time  6ti  and  the  order 
of  activation  is  given  by  Sn.  Two  cases  are  examined: 

1.  Periodic.  The  switching  sequence  is  periodic  of  length  N  and  is  a  repetition  of  some 
s  G  Sm- 

2.  General.  The  switching  sequence  is  a  concatenation  of  members  of  Sn- 

The  stability  results  come  directly  from  those  for  time-varying  systems  [150].  We  note  that 
in  the  case  of  controllability,  for  example,  the  rank  of  N\  matrices  must  be  tested.  Further, 
the  restriction  to  constant  SU  is  quite  restrictive. 
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§2.2.3  SYSTEMS  WITH  IMPULSE  EFFECT 

Closely  related  to  hybrid  systems  is  the  work  on  so-called  systems  with  impulse  effect. 
Most  of  these  results  appear  only  in  the  Russian  literature,  but  fortunately  there  is  an 
English  monograph  [10]  which  summarizes  major  results. 

A  system  of  differential  equations  with  impulse  effect  is  given  by 

1.  the  system  of  differential  equations 

x{t)  =  f{t,x),  (2-3) 

where  t  €  M,  x  €  C  M" ,  /  :  R  x  ^  M  ; 

2.  sets  Mu  Nt  of  arbitrary  topological  structure  contained  in  R  x  fi; 

3.  the  operator  At  •  Mt  Nt- 

The  motion  begins  from  {to,xo)  and  moves  along  the  curve  (t,x(t))  described  by  Equa¬ 
tion  (2.3)  with  initial  condition  x(to)  =  xo  until  the  instant  n  >  to  when  it  meets  the  set 
Mt  At  time  n  the  operator  instantly  transfers  the  point  from  position  (ti,x(ti))  to 
'  (n  x+)  eNr  ,xi  =  AtiX(ti),  from  which  the  process  continues  to  evolve  under  Equation 
(2.3)  until  it  again  encounters  Mu  etc.  Three  classes  of  systems  are  considered  in  [10]. 

•  Class  I:  Systems  with  fixed  instants  of  impulse  effect  are  those  for  which  Mt  is 
represented  by  a  sequence  of  hyperplanes  t  =  n,  where  {r^}  is  a  given  sequence  of 
instants  of  impulse  effect.  In  this  case,  At  is  only  defined  for  t  -  Tk,  giving  a  sequence 
of  operators 

•  Class  II:  Systems  with  mobile  instants  of  impulse  effect  are  those  for  which  Mt 
is  represented  by  a  sequence  of  hypersurfaces  <Tk=t  =  Tfc(x),  k  £  Z+.  It  is  assumed 
that  Tk{x)  <  Tfc+i(x)  for  X  e  0  and 

lim  Tfc(x)  =  00 

k-¥oo 

for  all  X  €  Again,  At  restricted  to  hypersurface  an  is  given  by  operator  Ak- 

•  Class  III:  Autonomous  systems  with  impulse  effect  are  those  where  the  sets 
Mt  =  M  and  Nt  =  N  axe  subsets  of  fi  and  At  s  A  :  M  iV.  Further,  the  equation 
f{t,x)  is  replaced  by  g(x). 

Class  III  systems  are  closely  related  to  hybrid  systems,  as  we  see  below.  Virtually  all 
the  results  in  [10],  however,  are  for  Class  I  and  II  systems. 

§2.2.4  IMPULSE  CONTROL 

Impulse  control,  or  control  by  interventions,  deals  with  choosing  a  discrete  strategy 
for  controlling  a  set  of  differential  equations  with  minimal  cost.  In  particular,  consider 

X{t)  =  /(x(t)),  x{to)  =  XQ, 

where  x{t)  €  R",  and  let  ^{t,xo)  denote  its  solution.  There  is  a  running  cost  A: :  R"  R+ 
associated  with  the  continuous  state. 
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The  discrete  dynamics  is  modeled  by  a  set-valued  map  F  :  R”  — >  2^^  of  destinations, 
each  with  an  associated  cost.  Let  Grp  =  |  x  G  IR",u  €  r(x)}.  It  is  assumed  that 

r(x)  is  compact  for  each  x  and  that  the  jump  cost  c  :  Grp  — t  IR+  is 

1.  bounded,  with  c(x,  •)  continuous  on  r(x), 

2.  c(x,v)  +  c(v,w)  >  c(x,  to.),  for  all  x  G  R" ,  v  €  r(x),  and  w  G  r(t;). 


3.  c(x,  u;)  >  c>  0,  for  all  x  G  R"  and  w  G  r(x). 

A  control  policy  tt  or  strategy  for  this  impulse  control  problem  is  a  sequence 
of  intervention  times,  intervention  states,  and  intervention  destinations,  where 
Wm  €  r(Xm). 

The  trajectory  under  tt  is  as  follows:  x{t)  =  ^{t,xo)  for  t  <  ti;  at  time  ti  the  state  is 
impulsively  transferred  from  xi  =  ^(ti,xo)  to  wi  at  a  cost  c(xi,tui);  from  there,  the  process 
continues. 

The  infinite-time,  discounted  cost  associated  with  the  above  problem  is 


J(7r,xo)=/  e  “*fe(x(t)) 
Jo 


where  a  >  0.  The  optimal  control  problem  is  to  minimize  J(7r,xo)  over  all  strategies.  For 
more  details  see  [51,  156] 


§2.2.5  PIECEWISE  DETERMINISTIC  PROCESSES 

A  piecewise-deterministic  process  (PDP)  taking  values  in  an  open  set  E  of  R"  with 
Borel  cr-field  and  boundary  dE  is  determined  by  [53] 

1.  the  flow  (f>{t,x)  in  E  satisfying  <^(t,^(s,x))  =  4>{t  +  s,xy, 

2.  the  jump  rate  A  :  -t  R+ ; 

3.  the  transition  probability  Q  :  B{E)  x  {EU  d*E)  [0, 1]  where 

d*E  =  {x  G  dJS  I  4>{-t,  z)  €E  for  t€  (0,  e)  some  e  >  0}.  (2.4) 


We  define  t*(x)  =  inf{f  >  0  |  ^(t, x)  G  d*E}  and  A(t,  x)  =  /q  A(0(s, x))  ds  for  0  <  t  <  t*{x). 

A  sample,  Xt  starting  from  x,  of  the  piecewise-deterministic  process  is  constructed  as 
follows.  Pick  a  number  Ti  from  the  distribution 


PATi>t)  = 


exp(-A(t,x)), 

0, 


t  <  t*{x), 
t  >  t*ix). 


Define  Xt  =  (l>{t,x)  for  t  <  Ti.  Now  independently  choose  an  fJ- valued  random  variable 
Zi  having  distribution  Q(-;  (/>(Ti,x))  and  define  Xt,  =  Zy.  The  process  now  restarts  at 
Zi.  It  is  assumed  that  A  is  bounded  and  the  sequence  of  jump  times  Ti,T2,...  satisfies 
lilUT^— foo  Pn  —  J^-a.s. 


^That  is,  B{E)  =  En  5(R" ). 
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§2.2.6  TIMED  AND  HYBRID  AUTOMATA 

There  is  a  growing  literature,  mainly  in  computer  science,  that  deals  with  hybrid  systems 
as  an  outgrowth  of  automata  theory.  The  main  idea  is  to  successively  add  time  constraints 
on  events  and  simple  dynamics  (such  as  clocks  and  timers)  to  finite  automata  in  order  to 
build  on  automata  results.  We  do  not  attempt  to  summarize  this  literature  and  refer  the 
interested  reader  to  [1,  3,  63,  66,  98,  120]. 

This  program  comes  full  circle  back  to  control  theory  in  the  work  of  Deshpande  and 
Varaiya  (also  cf.  [69]).  A  hybrid  automaton  [56,  57]  is  a  system 

i?  =  (Q,r,i:,£;,$,r), 

where  Q  is  the  finite  set  of  discrete  states,  1"  is  the  the  set  of  continuous  states,  and 
S  is  the  finite  set  of  discrete  events.  The  finite  set  of  edges, 

C  Q  X  2*”  X  S  X  {R”  R”}  X  Q. 

models  the  discrete  event  dynamics  of  the  system.  An  edge 

E  3  €  =  (Qci  Ae,  Vei  Qe) 

is  enabled  when  the  discrete  state  is  in  Qe  and  the  continuous  state  is  in  X^.  When  the 
transition  is  taken,  the  event  14  €  E  is  accepted,  the  continuous  state  is  reset  according  to 
map  re,  and  the  system  enters  discrete  state  9' .  The  reset  map  is  allowed  to  be  set  valued 
(denoted  Re),  in  which  case  the  continuous  state  is  reset  nondeterministically  when  e  is 
taken.  The  continuous  dyneimics  is  given  by  the  set  of  controlled  vector  fields 

$  =  {F,  :  R"  2'*"\{0}  U  €  Q}. 

When  the  discrete  state  is  q,  the  continuous  state  evolves  according  to  the  differential 
inclusion 

Xc{t)  G  FgiXcit)). 

Finally, 

r  =  {F,  C  R"  I  g  €  Q} 

is  a  set  of  invariance  conditions  that  the  system  state  must  satisfy,  that  is,  when  the  system 
is  in  phase  q,  the  continuous  state  is  required  to  be  in  Fg. 


Part  I 


Modeling  of  Hybrid  Systems 


Chapter  3 

Hybrid  Phenomena  and  Models 


In  this  chapter,  we  identify  the  discrete  phenomena  which  occur  in  hybrid  systems.  We  then 
review  in  detail  several  models  of  hybrid  systems  from  the  control  and  dynamical  systems 
literature  and  draw  relations  among  them. 

§3.1  INTRODUCTION 

A  hybrid  system  is  a  system  that  involves  continuous  s^tes  and  dynamics,  as  well  as  some 
discrete  phenomena  corresponding  to  discrete  states  and  dynamics.  In  this  thesis,  our 
focus  is  on  the  case  where  the  continuous  dynamics  is  given  by  a  differential  equation 

i(t)  =  ^(t),  t  >  0.  (3.1) 

Here,  x{t)  is  the  continuous  component  of  the  state  talcing  values  in  some  subset  of  a  Eu¬ 
clidean  space.  C(t)  is  a  vector  field  which  generally  depends  on  x{t)  and  the  aforementioned 
discrete  phenomena. 

Hybrid  control  systems  are  control  systems  that  involve  continuous  states,  dynamics, 
and  controls,  as  well  as  discrete  phenomena  corresponding  to  discrete  states,  dynamics,  and 
controls.  Here,  ^{t)  is  a  controlled  vector  field  which  generally  depends  on  x{t),  the  contin¬ 
uous  component  u{t)  of  the  control  policy,  and  the  aforementioned  discrete  phenomena. 

The  chapter  is  organized  is  follows.  First,  we  begin  by  identifying  the  discrete  phenom¬ 
ena  that  generally  arise  in  hybrid  systems.  We  identify  four  types: 

1.  autonomous  switching, 

2.  autonomous  jumps  (also  called  autonomous  impulses), 

3.  controlled  switching, 

4.  controlled  jumps  (also  called  controlled  impulses). 

In  the  next  section,  we  briefly  examine  these  discrete  phenomenon  and  give  examples  of 
each.  We  also  discuss  how  digital  automata  may  be  viewed  as  evolving  in  continuous  time, 
which  sets  the  stage  for  their  interacting  with  ODEs  below. 

Next,  we  review  in  turn  several  models  of  hybrid  systems  developed  from  the  control 
and  dynamical  systems  point  of  view.  For  sure,  there  are  many  others  [3,  4,  66,  120].  The 
models  here  have  been  chosen  as  much  for  the  clarity  and  rigor  of  their  presentation  as  for 
the  mprhanifiTns  they  use  to  Combine  discrete  and  continuous  dynamics.  Specifically,  we 
review  the  following  models  of  hybrid  systems,  in  order  of  (original)  appearance  of  the  cited 
papers: 
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1.  Witsenhausen’s  model  [152,  §3.3], 

2.  Tavernini’s  model  [135,  §3.3], 

3.  Back-Guckenheimer-Myers  model  [9,  §3.5], 

4.  Nerode-Kohn  model  [114,  §3.6], 

5.  Antsaklis-Stiver-Lemmon  model  [5,  §3.7], 

6.  Brockett’s  models  [38,  §3.8]. 

For  further  discussion  and  examples,  the  reader  is  referred  to  the  original  papers. 

Some  models  in  the  papers  above  allow  time-varying  vector  fields,  but  we  only  consider 
time-invariant  ones  here.  Also,  we  have  sometimes  changed  notation  firom  the  original 
papers  to  make  the  presentation  more  uniform  and  place  the  models  in  as  similar  a  light  as 
possible.  In  §§3.9-3.10  we  briefly  discuss  and  compare  the  six  models. 

§3.2  HYBRID  PHENOMENA 

§3.2.1  AUTONOMOUS  SWITCHING 

Autonomous  switching  is  the  phenomenon  where  the  vector  field  ^(•)  changes  discon- 
tinuously  when  the  continuous  state  x{-)  hits  certain  “boundaries”  [5,  114,  135,  152].  The 
simplest  example  of  this  is  when  it  changes  depending  on  a  “clock”  which  may  be  mod¬ 
eled  as  a  supplementary  state  variable  [38].  An  example  of  autonomous  switching  is  the 
following. 

Example  3.1  (Hysteresis)  Consider  the  following  model  of  a  system  with  hysteresis  [135]: 

ii  =  X2-  <j>{xi), 

±2  =  H{lp{xi,X2))  -  4>{X2), 

where  the  multi-valued  function  H  is  shown  in  Figure  3-1.  The  functions  <j),  depend  on 
the  exact  system  under  consideration. 

H 


Figure  3-1:  Hysteresis  function. 

Note  that  this  system  is  not  just  a  differential  equation  whose  right-hand  side  is  piecewise 
continuous.  There  is  “memory”  in  the  system,  which  affects  the  value  of  the  vector  field. 
Indeed,  such  a  system  natrirally  has  a  finite  automaton  associated  with  the  function  H,  as 
pictured  in  Figure  3-2. 
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[a  <  Ip  <  d] 


Figure  3-2:  Finite  Automaton  Associated  with  Hysteresis  Function. 


§3.2.2  AUTONOMOUS  JUMPS 

An  autonomous  jump  is  the  phenomenon  where  the  continuous  state  x{-)  jumps  discon- 
tinuously  on  hitting  prescribed  regions  of  the  state  space  [9,  10].  We  may  also  call  these 
autonomous  impulses.  The  simplest  examples  possessing  this  phenomenon  are  those 
involving  collisions. 

Example  3.2  (Collisions)  Consider  the  case  of  the  vertical  and  horizontal  motion  of  a 
ball  of  mass  m  in  a  room  under  gravity  with  constant  g  {see  Figure  3-3).  In  this  case,  the 
dynamics  are  given  by 


X  =  Vx, 

y  =  Vy, 

Vx  ~  0, 

Vy  =  -mg. 


Further,  upon  hitting  the  boundaries  {(z,  y)  |  y  —  0  or  y  —  C}  we  instantly  set  Vy  to  pvy, 
where  p  €  [0, 1]  is  the  coefficient  of  restitution.  Likewise,  upon  hitting  {(z,  y)  j  z  =  0  or  z  = 
R}  Vx  is  set  to  —pvx- 


§3.2.3  CONTROLLED  SWITCHING 

Controlled  switching  is  the  phenomenon  where  the  vector  field  changes  abruptly  in 
response  to  a  control  command,  usually  with  an  associated  cost.  This  can  be  interpreted 
as  switching  between  different  vector  fields  [156].  Controlled  switching  arises,  for  instance, 
when  one  is  allowed  to  pidc  among  a  number  of  vector  fields; 

i  =  /t(®))  i  €  Qci  {l,2,...,iV}. 

Example  3.3  (Satellite  Control)  As  a  simple  example  of  satellite  control  consider 

0  =  TeffV, 

where  9  is  angular  position,  9  angular  velocity  of  the  satellite,  and  v  €  {-1,0,1}  depending 
on  whether  the  reaction  jets  are  full  reverse,  off,  or  full  on. 
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Figure  3-3:  Ball  bouncing  in  an  enclosed  room. 


An  example  that  includes  controlled  switching  and  continuous  controls  is  the  following. 

Example  3.4  (Transmission)  Consider  a  simplified  model  of  a  manual  transmission, 
modified  from  one  in  [38]: 

Xl  =  X2, 

X2  =  [-a{x2lv)  +  u\l{l  +  v), 

where  xi  is  the  ground  speed,  X2  is  the  engine  RPM,  u  €  [0, 1]  is  the  throttle  position,  and 
V  e  {1,2, 3, 4}  is  the  gear  shift  position.  The  function  a  is  positive  for  positive  argument. 

§3.2.4  CONTROLLED  JUMPS 

An  controlled  jump  is  the  phenomenon  where  the  continuous  state  x(-)  changes  discon- 
tinuously  in  response  to  a  control  command,  usually  with  an  associated  cost  [14].  We  may 
also  call  these  controlled  impulses.  An  example  is  the  following. 

Example  3.5  (Inventory  Management)  In  a  simple  inventory  management  model  [14], 
there  are  a  “discrete”  set  of  restocking  times  Ox  <  62  <•-•  and  associated  order  amounts 
ai,a2, _  The  equations  governing  the  stock  at  any  given  moment  are 

y{t)  =  —p{t)  + 

i 

where  fi  represents  degradation  or  utilization  dynamics  and  5  is  the  Dirac  delta  function. 


NOTE.  If  one  makes  the  stocking  times  and  amounts  an  explicit  function  of  y  (or  t),  then 
these  controlled  jumps  become  autonomous  jumps. 
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§3.2.5  DIGITAL  AUTOMATA  AND  ODEs 

Usually,  digital  automata  [§2.1.4]  are  thought  of  as  evolving  in  “abstract  time,”  where  only 
the  ordering  of  symbols  or  “events”  matters: 

Qk+i  = 

Ok  =  v{Qk,ik)- 

We  may  add  the  notion  of  time  by  associating  with  the  feth  transition  the  time  at  which  it 
occurs: 

q{tk+i)  =  i'{q{tk),i{tk)), 
o(tk)  =  T){q{tk),i{tk))- 

Finally,  this  automaton  may  be  thought  of  as  operating  in  “continuous  time”  by  the  con¬ 
vention  that  the  state,  input,  and  output  symbols  are  piecewise  right-  or  left-  continuous 
functions. 

NOTE.  The  notation  t~  may  used  to  indicate  that  the  finite  state  is  piecewise  continuous 
from  the  right: 

q{t)  =  u{x{t),q{t~)) 

Likewise 

qif*-)  =  u{x{t),q{t)) 

denotes  that  it  is  piecewise-continuous  from  the  left.  To  avoid  making  the  distinction  here 
we  use  Sontag’s  more  evocative  discrete-time  transition  notation  [133] 

q+{t)  =  v{x{t),q(t)) 

to  denote  the  “successor”  of  q{t).  Its  “predecessor”  is  denoted  q~it).  This  notation  makes 
sense  since  no  matter  which  convention  is  used  for  q{tys  piecewise  continuity,  we  still  have 
q+it)==q{t+). 

The  result  is  the  following  system  of  equations,  where  q,  o  are  piecewise  continuous  in  time: 

q+{t)  =  t^{q{t),i{t)),  /3  2) 

o{t)  =  T]{q{t),i{t)). 

Here,  the  state  q{t)  changes  only  when  the  input  symbol  i{t)  changes.  Also,  note  that  this 
reduces  to  the  previous  automaton  equation 

q+[t]  =  l^{q{[t]),i[^]), 

oW  = 

where  [t]  denotes  the  time  at  which  the  input  symbol  last  changed.  Thus,  we  have  the 
idea  of  an  automaton  whose  update  times  are  not  the  abstract  members  of  Z+,  but  the 
event  times  in  E  when  the  input  symbol  i  changes.  Note  that  in  the  usual  case,  a  finite 
automaton  can  be  presented  with  the  same  input  symbol  for  two  successive  time  intervals. 
These  situations  must  be  handled  differently  (e.g.,  by  adding  new  states  and  symbols)  in 
the  continuous-time  version. 

Finally,  we  may  wish  to  model  the  interaction  of  digital  automata  with  ODEs.  To 
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accomplish  this  we  note  that  such  equations  can  be  thought  of  as  a  special  case  of  ODEs 
with  controlled  jumps  (§4): 

q{t)  =  0, 

9+(t)  =  u{q{t),i{t)), 
o{t)  =  77(9(0)  *(<))• 

Suppose  these  equations  interact  with  a  set  of  differential  equations  with  output  i{t). 
Thus,  the  symbol  i{t)  jumps  as  a  function  of  the  state  x{t)  of  the  differential  equation 

a:(0  =  f{x{t),u{t)), 

i{t)  =  h{x{t)). 

At  these  interaction  times,  the  discrete  phenomena  are  autonomous  jumps  in  the  combined 
system.  We  may  also  allow  u  to  depend  on  o{t),  which  is  the  case  in  hybrid  systems  below. 

§3.3  WITSENHAUSEN’S  MODEL 

In  [152],  Witsenhausen  introduces  a  class  of  continuous-time  systems  with  part  continuous, 
part  discrete  state — in  short,  what  we  would  call  a  continuous-time  hybrid  system.  The 
class  of  systems  he  considers  is  restricted  by  the  following  conditions: 

1.  At  a  transition,  that  is,  at  a  time  when  the  discrete  state  undergoes  a  change,  the 
state  vector  is  still  continuous,  though  the  vector  field  niay  change  discontinuously. 
No  jumps  of  the  continuous  state  vector  are  allowed. 

2.  Transitions  occur  when  and  only  when  the  continuous  state  vector  satisfies  a  condi¬ 
tion  given  for  each  type  of  transition.  In  the  case  of  control  inputs,  these  influence 
transitions  only  through  the  differential  equations,  never  directly. 

3.  Some  technical  requirements  on  the  data  describing  the  system. 

Witsenhausen  starts  with  continuous  systems,  described  by 

x{t)  =  f{x{t),u{t)), 

where  x{t)  €  R”  and  u{t)  €  R”*.  The  effect  of  the  discrete  components  of  the  state  on  these 
continuous  components  is  accounted  for  by  letting  /  depend  on  the  discrete  state: 

=  /(q(t),x(t),u(t)). 

The  coupling  in  the  opposite  direction  is  described  by  conditions  on  the  continuous  state 
for  which  transitions  of  the  discrete  component  occur.  In  particular,  let  Q  C  Z  be  finite. 
The  state  of  the  system  is  characterized  at  any  time  by  the  pair  (q,  x)  ranging  over  Q  x  R"; 
q  is  called  the  discrete  state,  and  x  is  the  continuous  state. 

A  transition  of  the  discrete  state  from  q  =  i  to  q  =  j^i^  triggered  when  the 
continuous  state  ,x  reaches  a  given  transition  set  Mij  C  R"  ■  There  is  one  such  set, 
possibly  empty,  for  each  ordered  pair  of  distinct  indexes  from  Q. 

Define  the  arrival  set 

Mt  =  [j 
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That  is,  the  set  of  all  values  of  the  continuous  state  for  which  a  transition  into  discrete 
state  i  can  occur  from  some  other  discrete  state  (in  our  parlance,  the  set  of  destinations  of 
autonomous  switchings  into  discrete  state  i).  Likewise,  define  the  departure  set 

That  is,  the  set  of  all  values  of  the  continuous  state  for  which  a  transition  from  discrete 
state  i  into  some  other  discrete  state  can  occur  (in  o\ir  parlance,  the  set  of  autonomous 
switchings  from  discrete  state  i). 

Witsenhausen  places  three  key  assumptions  on  the  sets  above,  which  we  explore  in  some 
detail: 

1.  For  any  three  distinct  indexes  i,j,k  from  Q,  the  sets  Mij  and  Mi^k  disjoint. 

2.  For  all  i  €  Q,  M~  is  closed  in  1”  (equipped  with  the  usual  topology). 

3.  For  all  i  €  Q,  the  sets  and  M~  are  disjoint  in  R". 

The  first  assumption  says  that  it  is  never  the  case  that  the  conditions  for  transition  to  two 
or  more  different  discrete  states  are  fulfilled.  The  second  assumption  is  more  technical;  its 
usefulness  is  seen  in  the  description  of  the  dynamics  below.  The  third  assumption  guarantees 
that  after  a  discrete  transition  is  made,  the  condition  for  a  further,  instantaneous  transition 
to  some  other  discrete  state  does  not  occur.  This  rules  out  “transition  loops,”  in  which  an 
infinite  number  of  discrete  transitions  occur  in  a  single  instant.  If  there  is  no  loop,  and  a 
transition  sequence  starting  at  (x,io)  goes  through  (x,ti), . . .  and  terminates  at  some  state 
{x,i*),  then  the  whole  sequence  can  be  avoided  by  considering  x  €  instead  of 

Witsenhausen  assumes  that  each  f{q,x,u)  is  continuous  in  {x,u)  for  fixed  q  and  con¬ 
tinuously  differentiable  in  x  for  fixed  iq,u).  He  restricts  the  set  of  controls  to  be  as  follows. 
Let  tf  be  a  variable  end  time  in  [fo,oo)  and  C  R"*,  closed.  A  control  is  a  pair  [t/,u(-)] 
where  u(-)  is  any  piecewise  continuous  function  from  [to,tf]  into  Q. 

We  are  now  ready  to  describe  the  evolution  of  the  system  from  initial  state  (xo,9o)  at 
time  to  until  (xf,q/)  at  time  tf,  imder  the  control  [t/,u(-)]: 

1.  Solve 

i(i)  =  f(9o,^(i)i  u(t)),  x(to)  =  xo- 

The  conditions  on  /  assure  the  existence  of  a  unique  solution  either  up  to  </  or  up  to 
some  earlier  escape  time.  In  either  case  a  continuous  path,  xo(-)i  in  R"  is  given.  If 
this  path  does  not  meet  M~,  then  it  gives  the  evolution  of  the  continuous  state  and 
the  discrete  state  remains  constant  at  qq, 

2.  If  the  path  xo(-)  meets  M~  at  some  point  x*  at  time  t*,  then  by  virtue  of  xo([to,**]) 
compact  and  M~  closed  (Assumption  2),  there  exists  an  earliest  time  of  intersection 
ti  with  corresponding  point  ii.  By  Assumption  1,  xi  belongs  to  exactly  one  of  the 
transition  sets  in  M~,  say,  Mqo,qi-  The  state  of  the  system  at  time  <i  is  defined  to  be 
(xi,gi).  Note  that  ti  >  to,  since  otherwise  (xo,gi)  would  have  been  considered  the 
initial  state. 

3.  The  point  xi  now  belongs  to  and  (by  Assumption  3)  not  to  M~.  Thus,  we  may 
repeat  the  above  steps  from  (xi,  qi).  Furthermore,  if  a  second  transition  time  <2  exists, 
it  satisfies  *2  >  ti- 
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Thus,  the  above  rules,  applied  recursively,  generate  a  unique  state  evolution  [x(-),g(-)]  up 
to  t/,  some  earlier  escape  time,  or  a  critical  time  representing  a  point  of  accumulation  of 
transition  times.  See  Figure  3-4. 

Finally,  Witsenhausen  notes  that  since  the  transitions  of  a:(-)  always  take  place  on  the 
boundary  dM~  of  a  departure  set  (which  is  closed  by  Assumption  2),  all  transition  sets  Mij 
can  be  replaced  by  their  reductions  MijDdM'.  He  also  gives  some  optimal  control  results, 
which  are  discussed  in  §10.  We  refer  to  the  above  as  the  WHS  model,  for  Witsenhausen 
hybrid  systems. 

§3.4  TAVERNINI’S  MODEL 

Tavernini  introduces  and  discusses  so-called  differential  automata  in  [135].  He  was  moti¬ 
vated  to  study  such  systems  as  a  means  of  modeling  hysteretic  phenomena  like  backlash 

and  friction  (cf.  Example  3.1). 

A  differential  automaton.  A,  is  a  system  (5,/,!/)  where 

•  5  is  the  state-space  of  A,  5  =  R"  x  Q,  Q  ~  {1, 2, . . . ,  W}  is  the  discrete  state  space 
of  A,  and  R"  is  the  continuous  state  space  of  A; 

•  /  is  a  finite  family  f{-,q)  :  R"  -)•  R",  g  €  Q,  of  vector  fields,  the  continuous 
dynamics  of  A; 

•  1/  :  S  Q  is  the  discrete  transition  function  of  A. 

Let  Ug  =  v{-,q),  q  €  Q.  Define  I{q)  =  i/,(R")\{g},  that  is,  the  set  of  discrete  states 
“reachable  in  one  step”  from  q.  We  require  that  for  each  q  e  Q  and  each  p  €  /(g)  there 
exist  closed  sets 

Mg,p^i'g^ip)- 

The  sets  dMg^p  are  called  the  switching  boundaries  of  the  automaton  A.  Define  M,  = 
Up6/(9)  -^9.?  define  the  domain  of  capture  of  state  g  by 

C(g)  =  R”\M,  =  {x  G  R”  I  i^(x,g)  =  g}. 

The  equations  of  motion  axe 

i{t)  =  f{x{t),q{t)), 

q'^it)  =  i^(a:(t),g(i)), 

with  initial  condition  [x(0),  g(0)]^  G  UgeQ  ^{q)  ^  {9}-  The  notation  t  indicates  that  the 
discrete  state  is  piecewise  continuous  from  the  right.  Thus,  starting  at  [xo,  *],  the  continuous 
state  trajectory  x(-)  evolves  according  to  x  =  f{x,i).  If  x(-)  hits  some  dMij  at  time  h, 
then  the  state  becomes  [x(fi),  j],  from  which  the  process  continues.  See  Figure  ^4. 

Tavernini  places  restrictions  on  the  model  above:  First  each  f{',q)i  g  €  Q,  is  assumed 
to  be  globally  Lipschitz  so  that  the  continuous  dynamics  are  well-behaved.  Also,  for  each 
g  G  Q  and  p  G  /(g),  the  set  Mg,p  is  required  to  be  connected  and  there  must  exist  a  function 
gg,p  e  C^R",  R)  with  0  in  its  image  a  regular  value  such  that 

Mg,p  =  {x  G  R”  I  g,,p(x)  >  0}. 
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Figure  3-4:  Example  dynamics  of  Tavernini’s  model. 


Thus,  Vq^ip)  is  an  n-submanifold  of  R"  with  boundary 

dMq^p  =  {x  €  R"  I  9g,p{x)  =  0}, 

which  is  an  (n  —  l)-sub manifold  of  R" . 

Finally,  [135]  makes  the  following  three  key  assumptions  on  differential  automata: 

•  Define  a,  =  min{dist(M,,p,  Mjy)  ]  p,p'  G  /(g), p  #  p'}.  We  require  that 

a  (A)  =  min  O',  >  0 

be  satisfied.  That  is,  the  distance  between  any  two  sets  with  different  discrete  transi¬ 
tions  is  bounded  away  from  zero. 

•  Define  /3,,p  =  min{dist{dMq,p,dMp,p>)  \  p'  €  I(p)}.  We  require  that  the  inequality 

/3{A)=mm  mn  /?,,p  >  0 

96<3  p€l{g) 

be  satisfied.  That  is,  after  a  discrete  transition,  the  next  set  from  which  another 
discrete  transition  takes  place  is  at  least  a  fixed  distance  away. 

•  The  assumption  on  oc{A)  is  such  that  C(g)  is  an  open  set  with  boundary  5C(g)  = 
dMq  =  Upe/(q)  dMq^p.  We  require  that  the  inclusions 

dMq,pCC(p),  peI{qU£Q 

be  satisfied.  That  is,  after  a  discrete  transition  one  is  found  in  an  open  set  on  which 
the  dynamics  are  well-defined. 

With  these  assumptions,^  Tavernini  proves  that  the  initial  value  problem  has  a  unique 
solution  with  finitely  many  switching  points.  Let  [x(-),g(-)]  denote  the  solution  correspond¬ 
ing  to  the  initial  value  [xo,go]'  Then  g  defines  a  sequence  of  discrete  states 


*  Actually,  the  vector  fields  f{-,q)  and  switching  functions  p,,p  are  assumed  to  be  smooth. 
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with  switching  points  where  U  denotes  the  time  of  transition  from  qi-i  to  If 

[x'(*),g'(*)]  denotes  the  solution  when  the  initial  value  is  [xq^qq]  where  Xq  is  near  xo,  then 
we  should  have  [x'(-),9'(-)]  “near”  [x(-),9(-)]  in  the  sense  that  q'  should  define  the  same 
“discrete  trajectory”  90)9i)92)--  -  with  possibly  different  switching  points.  However,  the 
corresponding  switching  points  of  the  two  solutions  should  be  close,  i.e.,  \t'^  —  tt|  should  be 
“small”  whenever  |a;g  —  rol  is  “small.”  Tavernini  defines  a  topology  to  make  this  precise. 
He  also  shows  that  the  set  of  points  with  such  a  property  is  an  open,  dense  subset  of  C{qo), 
denoted  5°. 

Finally,  Tavernini  concentrates  on  the  analysis  of  numerical  approximations  of  the  tra¬ 
jectories  of  differential  automata.  Briefly,  if  (xo,go)  G  5°  then  the  result  of  numerical 
integration  of  the  trajectory  starting  firom  x'q  uniformly  approaches  that  of  the  differential 
automaton  starting  firom  xg  as  the  integration  step  size  plus  d(xo,a:g)  goes  to  zero.  See 
[135]  for  details. 

We  refer  to  the  above  as  the  TDA  model,  for  Tavernini’s  differential  automata. 


§3.5  BACK-GUCKENHEIMER-MYERS  MODEL 

The  framework  proposed  by  Back,  Guckenheimer,  and  Myers  in  [9]  is  similar  in  spirit  to 
the  Tavernini  model.  The  model  is  more  general,  however,  in  allowing  “jumps”  in  the 
continuous  state-space  and  setting  of  parameters  when  a  switching  boundary  is  hit.  This  is 
done  through  transition  functions  defined  on  the  switching  boundaries.  Also,  the  model 
allows  a  more  general  state  space. 

More  specifically,  the  model  consists  of  a  state  space 

where  each  Xq  is  a  connected,  open  set  of  R” .  Notice  that  the  sets  Xq  are  not  required  to 
be  disjoint. 

The  continuous  dynamics  are  given  by  vector  fields  /,  :  X,  ->  R”.  Also,  one  has  open 
sets  Uq  such  that  Uq  C  Xq  and  dUq  is  piecewise  smooth.  For  g  €  Q,  the  transition  functions 

Gq.Xq^XxQ 

govern  the  jumps  that  take  place  when  the  state  in  Xq  hits  dUq.  They  must  satisfy 
Tti{Gq{x))  e  U^^^GJx)),  where  tt*.  is  the  kth.  coordinate  projection  function.  Thus,  TTi{Gq{x)) 
is  the  “continuous  part”  and  7^2(0 q{x))  is  the  “discrete  part”  of  the  transition  function. 

The  dynamics  are  as  follows.  The  state  starts  at  point  xq  in  Ui-  It  evolves  according  to 
x  =  fi{x).  If  a:(-)  hits  some  dUi  at  time  ti,  then  the  state  instantaneously  jumps  to  state  ^ 
in  Uj,  where  Gi(a:(ti))  =  (^,  j).  Prom  there,  the  process  continues.  We  refer  to  this  as  the 
BGM  model.  See  Figure  3-5,  which  is  taken  firom  [9]. 

As  in  [135],  it  is  assumed  in  [9]  that  the  switching  boundaries  are  fairly  regular.  In  par¬ 
ticular,  it  is  assumed  that  the  switching  boundaries  dUq  have  a  concrete  representation 
in  terms  of  the  zeros  of 

hq  =  min{/i,,i, . . . , 

where  the  hq^i :  X,  -4  R  are  smooth.  The  convention  then  is  such  that  /i,  >  0  on  Uq.  Thus, 
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Figure  3-5:  Example  dynamics  of  Back-Guckenheimer-Meyers  model.  Reproduced  from  [9]. 


the  switching  boundaries  are  (n  —  l)-dimensional  Lipschitz  continuous  manifolds. 

NOTE.  This  does  not  add  much  power  (over  single  functions)  since  Lipschitz  functions  are 

strongly  approximated  by  functions:  for  every  e  >  0  a  function  can  be  chosen  that 

coincides  with  a  Lipschitz  function  except  on  a  set  of  measure  e  [109]. 

The  model  above  is  fairly  expressive,  allowing  the  modeling  of  a  large  variety  of  phenom¬ 
ena.  However,  its  expressiveness  does  allow  the  possibility  of  some  seemingly  “anomalous” 
behavior.  For  example,  since  one  allows  jumping  to  the  boundary  of  the  sets  Uu  trajectories 
may  infinitely  “cycle”  if  Gj(x)  =  (y,i)  and  Gi{y)  =  (i,  j).  In  a  simulation  facility,  however, 
such  conditions  can  presumably  be  detected  and  reported  to  the  user  for  interpretation. 

The  paper  [9]  presents  computer  tools  that  have  been  developed  by  its  authors  for  the 
simulation  of  hybrid  systems.  As  an  example,  Raibert’s  one-legged  hopping  robot  [122]  is 
placed  into  their  framework. 

We  refer  to  the  above  model  (simplified  from  the  one  in  [9])  as  the  BGM  model. 

§3.6  NERODE-KOHN  MODEL 

In  [114],  Nerode  and  Kohn  take  an  automata-theoretic  approach  to  systems  composed  of 
interacting  ODEs  and  FA.  The  basic  philosophy  of  the  models  discussed  in  [114]  is  given 
in  great  generality,  with  a  subsequent  specialization  to  various  cases,  e.g.,  deterministic 
versus  non-deterministic.  To  keep  the  discussion  germane  to  that  so  far,  we  discuss  here 
the  so-called  “event-driven,  autonomous  sequential  deterministic  model”  [114,  p.  331].  We 
refer  to  it  as  the  NKSD  (for  sequential  deterministic)  model.  Here,  autonomous  refers  to 
the  fact  that  the  ODEs  do  not  explicitly  depend  on  time,  although  this  is  without  loss  of 
generality  by  appending  to  the  state  a  single  equation  for  f. 

The  model  consists  of  three  basic  parts:  plant,  digital  control  automaton,  and  interface. 
In  turn,  the  interface  is  comprised  of  an  analog-to-digital  (AD)  converter  and  digital-to- 
analog  (DA)  converter.  See  Figure  3-6. 

The  plant  is  modeled  as  in  Equation  (2.2) It  is  considered  to  be  an  input/output 
automaton  in  the  following  sense.  The  states  of  the  system  (in  this  sequential  deterministic 
case)  are  merely  the  usual  plant  states,  members  of  R”  [114,  p.  333].  The  input  alphabet  is 


*We  have  lumped  the  control  and  disturbance  signals  of  [114]  into  a  single  signal  u. 


62  Hybrid  Phenomena  and  Models 


Figure  3-6:  Hybrid  system  as  in  Nerode-Kohn  model. 


formally  taken  to  be  the  set  of  members  of  {u{-),6k)  where  6k  is  a  positive  scalar  and  «(•)  is 
a  member  of  the  set  of  piecewise  right-continuous  functions  in  Let  PU^  for  piecewise 

C7,  denote  the  latter  set.  Suppose  the  plant  is  in  state  Xk  at  time  tk-  The  “next  state”  of  the 
transition  function  from  this  state  with  input  symbol(u(-),  6k)  is  given  by  Xk+i  =  x{tk  +  6k), 
where  i(-)  is  the  solution  on  [4,4  +  of 

x{t)  =  f{x{t),  u{t  -  4)),  x{tk)  =  Xk. 

Setting  4+1  =tk  +  6k,  the  process  is  continued. 

The  digital  control  automaton  is  a  digital  automaton  as  discussed  in  §2.1.4  and 
§3.2.5.  In  general,  then,  Q,  I,  and  O  are  each  isomorphic  to  subsets  of  Z+.  However,  the 
interesting  case  is  where  these  sets  are  finite,  which  is  discussed  below.  As  noted  before, 
this  automaton  may  be  thought  of  as  operating  in  “continuous  time”  by  the  convention 
that  the  state,  input,  and  output  symbols  are  piecewise  right-continuous  functions,  leading 
to  Equation  (3.2). 

It  remains  to  couple  these  two  “automata.”  This  is  done  through  the  interface  con¬ 
sisting  of  the  following  two  maps: 

•  The  analog-to-digital  map  AD  :Y  x  Q  I, 

•  The  digital-to-analog  map  DA  :  O  —¥  PU. 

The  AD  symbols  are  determined  by  (FA-state-dependent)  partitions  of  the  output  space 
Y.  These  partitions  are  not  allowed  to  be  arbitrary,  but  are  the  “essential  parts”  of  small 
topologies  placed  on  Y  for  each  g  €  Q.  We  explain  this  later.  To  each  o  €  O  is  associated 
an  open  set  of  PU.  The  DA  signal  corresponding  to  output  symbol  o  is  chosen  from  this 
open  set  of  plant  inputs.  The  scalar  4  is  a  formal  construct,  denoting  the  time  until  the 
next  “event.”  It  is  not  actually  computed  or  chosen  by  the  digital  automaton,  nor  is  it 
actively  used  by  the  plant  in  computing  its  update  equations. 

The  dynamics  of  the  above  model  a;re  then  similar  to  those  of  the  Tavernini  model. 
Two  important  distinctions  arise:  input  and  output  for  both  the  ODEs  and  FA  have  been 
included,  and  the  maps  AD  and  DA  have  been  added.  Specifically,  we  have 

at)  =  f[x{t),DA{o{t),t-[t])], 
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y{t)  =  /i[a:(t)], 
q+{t)  =  u[q{t),AD{yit),q{t))], 
o(t)  =  Ti[q{t),AD{y{t),q{t))]. 

Briefly,  the  combined  dynamics  is  as  follows.  Assiune  the  continuous  state  is  evolving 
according  to  the  first  equation  and  that  the  FA  is  in  state  q.  Then  AD{-,q)  assigns  to 
output  y{t)  a  symbol  from  the  input  alphabet  of  the  FA.  When  this  symbol  changes,  the  FA 
makes  the  associated  state  transition,  causing  a  corresponding  change  in  its  output  symbol 
o.  Associated  with  this  symbol  is  a  control  input,  DA{o),  which  is  applied  as  input  to  the 
differential  equation  until  the  input  symbol  of  the  FA  again  changes. 

Now,  we  explain  what  is  meant  by  the  “small  topologies”  mentioned  above,  concen¬ 
trating  on  the  AD  map.  Nerode  and  Kohn  introduce  topologies  that  make  each  mapping 
ADq  =  AD{-,q),  q  €  Q,  continuous  (see  Algorithm  6.2).  The  sets  AD~^{i),  i  E  I  are  the 
essential  parts  mentioned  above.  For  a  verification  that  ADq  is  continuous,  as  well  as  other 
results  on  AD  and  DA  maps,  see  [24,  §6]. 

The  starting  point  of  the  Nerode-Kohn  approach  is  an  assumption  that  one  can  only 
realistically  distinguish  points  up  to  knowing  the  open  sets  in  which  they  are  contained. 
That  is  what  led  them  to  use  the  small  topologies  above  to  encode  the  plant  output  symbols. 
However,  the  bottom  line  is  that  by  combining  information  of  inclusion  in  different  open 
sets,  the  ADq  functions,  q  E  Q,  form  partitions  of  the  measurement  space.  Although  the 
small  topologies  are  meant  to  provide  “reasonable  partitions,”  it  is  interesting  to  note  that 
one  can  still  “identify”  single  points  in  the  model. 

EXAMPLE.  Consider  as  a  representative  example  zero  in  [—1, 1].  Then  the  open  sets  [—1, 1], 
[—1,0),  and  (0,1]  give  information  to  exactly  deduce  r  =  0.  Such  anomalies  lead  to  a 
breakdown  of  the  description  of  the  dynamics  above  in  the  sense  that  it  is  easy  to  construct 
exsunples  where  the  formal  input  letter  to  the  plant  is  (u,0). 

The  Nerode-Kohn  paper  develops  the  underpinning  of  a  theoretical  framework  for  the 
hybrid  continuous/rule-based  controllers  used  by  Kohn  in  applications.  Continuity  in  the 
small  topologies  associated  with  the  AD  and  DA  maps  above  plays  a  vital  role  in  the  theory 
of  those  controllers.  See  [114]  and  the  references  therein  for  details. 

§3.7  ANTSAKLIS-STIVER-LEMMON  MODEL 

In  [5],  Antsaklis,  Stiver,  and  Lemmon  take  a  discrete-event  dynamical  systems  (BEDS,  [74]) 
approach  to  hybrid  systems.  Conceptually,  the  model  is  related  to  that  of  Nerode-Kohn, 
but  we  quickly  review  it  here.  We  refer  to  it  as  the  ASL  model. 

Like  the  NKSD  model,  the  ASL  model  consists  of  three  basic  parts:  the  plant,  the 
controller,  and  the  interface.  See  Figure  6-1.  The  plant  is  modeled  as  in  Equation  (2.2). 
The  controller  is  a  discrete  event  system,  modeled  as  a  digital  automaton.  We  think  of  it 
as  operating  in  continuous  time  as  in  §3.2.5: 

q+{t)  =  u{q{t),i{t)), 

o{t)  =  T]{q{t)), 

where  q{t)  E  Q,  i{t)  E  /,  and  o{t)  E  O,  the  state  space,  plant  symbols,  and  controller 
symbols,  respectively.  The  sets  Q,  /,  and  O  are  unspecified  in  [5],  but  we  take  from  context 
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that  they  are  each  isomorphic  to  subsets  of  Z-(-.  The  maps  are  u  :  Qx  I  Q  and  rj .  Q  O. 
The  subscript  k  denotes  the  fcth  symbol  in  a  sequence.  The  output  map  does  not  depend 
on  the  current  symbol,  which  is  without  loss  of  generality  after  adding  more  states. 

The  plant  and  controller  communicate  through  an  interface  consisting  of  two 
r>dess  maps,  AD  and  DA.  The  first  map,  called  the  actuating  function,  DA:O^W^, 
converts  a  controller  symbol  to  a  piecewise  constant  plant  input. 

u{t)  =  DA{o{t)). 

The  second  map,  called  the  plant  symbol  generating  function,  AD  :  R"  -t  /,  is  a  function 
which  maps  the  plant  state  space  to  the  set  of  plant  symbols  as  follows 

at)  =  AD{x{t)). 

The  function  AD  is  based  upon  a  partition  of  the  state  space,  where  each  element  of  the 
partition  is  associated  with  one  plant  symbol.  The  combined  dynamics  is  similar  to  that  of 

the  NKSD  model.  ,  .  n  j 

The  model  is  simple  but  fairly  general.  The  fact  that  arbitrary  partitions  are  allowed 

limits  what  one  can  prove  about  the  trajectories  of  this  model.  Several  example  systems 
are  given  in  [5].  Results,  mainly  from  the  BEDS  point  of  view,  may  be  found  in  [5]  and  the 
references  therein. 


§3.8  BROCKETT’S  MODELS 


Several  models  of  hybrid  systems  are  described  in  [38].  We  only  discuss  those  which  combine 
ODEs  and  discrete  phenomena  since  that  is  our  focus  here.  Two  models  combining  difference 
equations  and  discrete  phenomena  are  also  discussed  in  [38]. 

The  first  model,  which  Brockett  calls  a  type  B  hybrid  system,  is  as  follows: 


x{t)  =  f{x{t),u{t),v\p\), 

p{t)  =  r(i(t),tx(t),vbJ)> 


X  y 


%  and 


wherex(t)  e  X  CR",  u(i)  €  €R,  ubJ  €  y,  /  :K"  X 
r  :  R"  X  R”*  x  y  — >  R  Here,  X  and  U  are  open  subsets  of  R”  and  R  ,  respectively,  and  V  is 
isomorphic  to  a  subset  of  Z+.  Also,  the  rate  equation  r  is  required  to  be  nonnegative  for 
all  arguments,  but  need  have  no  upper  bound  imposed  upon  it.  We  denote  such  a  system 

as  BB,  short  for  Brockett’s  type  B  model.  .  ,  .  ^  .  , 

Brockett  has  mixed  continuous  and  “symbolic”  controls  by  the  mclusion  of  the  specif 
counter  variable  p.  The  control  u(t)  is  the  continuous  control  exercised  at  time  i;  the 
control  vlpj  is  the  pth  symbolic  or  discrete  control,  which  is  exercised  at  the  times  when 
p  passes  through  integer  values.  In  general,  one  may  also  introduce  (as  in  [38])  continuous 
and  symbolic  output  maps: 


y{t)  =  c(x(<),ubJ), 

oLpJ  =  »7(2/W.vbJ)- 


In  this  case,  one  may  limit  /  by  allowing  it  to  depend  only  on  y  instead  of  the  full  state  x. 
Note,  we  have  used  [t]  to  denote  the  value  of  t  at  which  p  most  recently  became  an  integer. 
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Brockett  also  introduces  a  type  D  hybrid  system  as  follows: 

x{t)  =  f{x{t),u{t),z[p\), 
p{t)  =  r{x{t),u{t),z\p\), 

z\p\  =  u{x[t],z\jp\,v[p\), 

where  z  €  Z,  and  Z  is  isomorphic  to  a  subset  of  Z+.  Here,  ^':lR"xZxV’  —^Z,  with  all 
other  definitions  as  above  except  that  Z  replaces  V  in  those  for  /  and  r.  Again,  u  and  v 
are  the  continuous  and  discrete  controls,  respectively.  We  denote  such  a  system  by  BD.  We 
may  picture  the  dynamics  as  in  Figure  3-7. 


Figure  3-7:  Example  dynamics  of  Brockett’s  Type  D  model. 

The  first  equation  denotes  the  continuous  dynamics  and  the  last  equation  the  “symbolic 
processing”  done  by  the  system.  The  times  when  p  passes  through  integer  values  can  be 
thought  of  as  the  discrete  event  times  of  the  hybrid  dynamical  system.  Thus,  we  consider 
BD  as  a  precise,  first-order  model  of  interactions  of  ODEs  and  BEDS.  Once  again  one  may 
introduce  output  equations: 

y{t)  =  c{x{t),z[p\), 

o[p\  =  ri(yM,2bJ)- 

Finally,  Brockett  generalizes  BD  to  the  case  of  “hybrid  system  with  vector  triggering” 
(herein,  BDV),  in  which  one  replaces  the  single  rate  and  symbolic  equations  with  a  finite 
number  of  such  equations: 

x{t)  =  f{x{t),u{t),z\p\),. 

Pi{t)  =  r{x{t),u{t),Zi\pi\), 

Zi\Pi]  =  Hx[t]pi,z{p\,Vi\pi\), 

where  i  €  {1, 2, ... ,  it}.  Again,  outputs  may  be  introduced. 
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In  order  for,  say,  the  BB  system  above  to  be  well-posed,  we  would  like  there  to  exist  a 
unique  solution  on  finite  interval  [0,r].  That  is,  given  [x(0),p(0)]  €  X  x  R,  there  should 
exist  unique  x(-)  and  p(-),  continuous  and  differentiable  almost  everywhere,  satisfying  the 
equations.  Brockett  meets  these  specifications  on  any  interval  in  which  p  does  not  take  on 
an  integral  value  by  requiring  /  to  be  Lipschitz  in  x,  continuous  with  respect  to  u,  and 
u  to  have  a  finite  number  of  discontinuities.  [It  is  also  necessary  to  assume  U  bounded]. 
The  result  extends  if  on  any  finite  interval  of  time  p  passes  through  only  a  finite  number  of 
integers,  leading  to  a  finite  number  of  discontinuities  of  the  derivatives  of  x  and  p  in  finite 
time.  In  general,  this  requires  similar  continuity  assumptions  on  r.  Consider,  for  example, 
the  case  where  V  =  Z+,  v[p\  =  ([pj  +1)^,  and  r  =  vLpJ.  This  leads  top  —  ([pj  +  1)  ,  which 
has  finite  escape  time.  Analogous  behavior  for  x  results  if  u  is  not  bounded.  In  the  usual 
case,  however,  U,  V,  and  Z  are  taken  compact,  avoiding  such  behavior.  Similar  discussion 
holds  for  models  BD  and  BDV. 

In  [38],  Brockett  gives  many  examples  of  systems  modeled  with  the  above  equations, 
including  buffers,  stepper  motors,  and  transmissions  (cf.  Example  3.4). 

§3.9  DISCUSSION  OF  REVIEWED  MODELS 

At  the  risk  of  oversimplification,  WHS,  TDA,  NKSD,  and  ASL  use  autonomous  switching, 
BGM  uses  autonomous  switching  and  autonomous  jumps;  and  BD  uses  a  combination  of 
autonomous  and  controlled  switching. 

Also,  comparing  to  systems  with  impulse  effect  [§2.2.3],  we  have  the  following.  BGM 
can  be  modeled  as  a  Class  III  system  (cf.  §5.2);  TDA,  and  autonomous  versions  of  WHS, 
NKSD,  and  ASL  [§3.10]  are  special  cases  of  Class  III  systems;  the  autonomous  versions  of 
BB  and  BD  [§3.10]  are  Class  II. 

From  the  control  perspective,  the  TDA  model  is  an  autonomous  system  and  the  BGM 
is  essentially  so  (although  one  can  set  parameters  on  jumps).  The  NKSD  and  ASL  models 
focus  on  the  “control  automaton,”  coding  the  action  of  the  controller  in  the  mappings  from 
continuous  states  to  input  symbols,  through  automaton  to  output  symbols,  and  back  to 
controls. 

Witsenhausen  adds  a  control  to  the  continuous  component  of  the  system  dynamics. 
Brockett ’s  BD/BDV  models  allow  the  possibility  of  both  continuous  and  discrete  controls  to 
be  exercised  as  input  to  the  continuous  and  symbolic  dynamics  of  the  systems,  respectively. 
That  is  the  plant  not  only  responds  to  the  state  (or  output)  of  the  finite  machine,  but  to 
continuous  commands  generated  separately  as  well.  One  may  argue  that  this  is  largely  a 
matter  of  level  of  modeling.  For  instance,  one  can  assume  (as  in  NKSD  and  ASL)  that  the 
“low-level”  loops  have  been  closed,  eliminating  the  continuous  control  from  the  design  of 
the  “high-level”  ones.  Nevertheless,  our  approach  in  this  thesis  is  more  in  spirit  with  those 
of  Witsenhausen  and  Brockett. 

Prom  the  original  papers,  it  is  clear  that  the  models  above  were  primarily  developed 
for  a  variety  of  purposes:  TDA  and  BGM  for  modeUng  and  simulation,  NKSD  and  ASL 
for  controlling  continuous  systems  with  computer  programs  or  “higher  level  controllers, 
and  Brockett’s  for  modeling  the  action  of  (hierarchical)  motion  control  systems.  Moreover, 
there  is  a  direct  trade-off  between  the  generality  of  a  model  and  what  one  can  prove  about 
such  a  mpdel.  Therefore,  “containment”  of  one  model  in  another  does  not  reflect  any  bias 
of  the  more  general  model’s  being  “superior.”  Indeed,  in  §5  we  develop  a  very  general, 
abstract  model  which  captures  many  hybrid  phenomena  and  the  models  reviewed  here. 
Later,  however,  we  place  restrictions  on  this  model  in  order  to  solve  a  related  control 
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problem  [§10]. 

§3.10  COMPARISON  OF  REVIEWED  MODELS 

In  the  sequel,  we  explore  the  capabilities  of  the  hybrid  systems  models  WHS,  TDA,  BGM, 
NKSD,  ASL,  and  BD,  described  above.  Clearly,  these  models  were  developed  for  different 
purposes  with  assumptions  arising  accordingly.  Nevertheless — and  for  expediency — we  note 
some  containment  relations  among  these  models. 

Here,  A  contains  B  means  that  every  system  described  by  the  equations  of  model  B 
can  be  described  by  the  equations  of  model  A.  When  the  equations  of  a  model  describe  a 
system,  we  say  that  the  model  implements  that  system. 

First,  since  we  are  not  interested  in  control  yet,  we  develop  autonomous  versions  of  the 
models  WHS,  NKSD,  ASL,  and  BD  above,  in  which  the  control  inputs  are  replaced  by  fixed 
functions  of  state.  For  instance,  an  autonomous  version  of  WHS  (denoted  WAUT)  arises 
by  dropping  dependence  of  the  vector  field  on  input  u{t). 

Next,  we  construct  an  autonomous  version  of  NKSD.  An  autonomous  version  of  ASL 
(denoted  ASLAUT)  can  be  constructed  similarly.  We  refer  to  the  following  as  NKAUT. 

x{t)  =  f{x{t),q{t)), 
q+{t)  =  v>{q{t),ADix{t),q{t))), 

where  x{t)  e  R”  and  q{t)  e  Q  ci  {1,. . .  ,N}.  Here,  /  :  R"  x  Q  ->•  R",  i/  :  Q  x  J  ->  Q,  and 
AD  :R"  x<5^7~{l,...,M}.  Note  that  we  have  incorporated  the  output  equations  into 
the  /,  u,  and  AD  functions.  The  AD  map  is  restricted  as  discussed  in  §3.6. 

Here  is  an  autonomous  version  of  the  BD  model,  which  we  refer  to  as  BAUT: 

x{t)  =  f{x{t),z\j)\), 

p{t)  =  r(z(t),zbJ). 

z\p]  =  u{x[tp\,z[p\,\p\), 

where  x{t)  €  R”,  p(t)‘e  R,  ^  ^  .AT},  /  :  R"  x  Z  -)•  R”,  r:  R”  x  Z  R,  and 

1/ :  R"  X  Z  X  Z  -f  Z.  As  in  BD,  r  is  restricted  to  be  normegative. 

By  construction,  the  original  models  contain  their  autonomous  versions.  Note  also  that 
BAUT  is  distinct  from  the  TDA  and  NKAUT  models  since,  for  instance,  it  allows  arbitrary 
dependence  of  the  discrete  dynamics  u  on  x  [tpj ,  which  can  lead  to  partitions  not  permitted 
by  the  other  two  models.  We  have  other  containment  relations  as  follows. 

Remark  3.6  BGM  contains  TDA. 

Proof.  Given  an  arbitrary  TDA  equation,  choose,  in  the  BGM  model,  A,  =  R” ,  Uq  — 
C{q)  =  R"\M„  /,  =  /(•,?),  Gq{x)  =  (x,p)  if  X  G  M,,p,  and  V  =  -p,,p  for  all  q  E  Q, 
pEl{q).  □ 

Remark  3.7  NKAUT  contains  TDA. 

Proof.  Suppose  we  are  given  an  arbitrary  TDA  equation  (i.e.,  a  differential  automaton 
A).  Let  primed  symbols  denote  those  in  the  NKAUT  model  with  the  same  notation  as 
those  for  the  differential  automaton.  Set  Q'  =  Q,  /'(•,?)  =  /(•,?),  q  e  Q'.  This  duplicates 
the  continuous  dynamics. 
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Now,  for  each  q  EQ',  choose  the  small  topology  on  R” 

r,  =  C(g)U  U  M‘p, 

peiiq) 


where  0  <  e  <  a(A)/3  and 

M'p  =  {x  €  R”|dist(x,  M,,p)  <  e}. 

The  non-empty  join  irreducibles  are  C{q)  and  M^p,  A*  p,  p  €  /(g),  where 

A‘,p  =  M‘p\M,.p. 

Let  ig^p,  jq,p,  kg  denote  the  symbols  associated  with  the  join  irreducibles  M*p,  A*_p,  and 
C{q),  respectively.  Defining 

^  {q'>jq,p)  ~ 

v'{q,kg)  =  g, 

duplicates  the  discrete  dynamics.  ^ 

Remark  3.8  BGM  contains  BA  UT. 

Proof.  Given  an  arbitrary  BAUT  equation,  choose,  in  the  BGM  model. 


Ui 

fl{x,q,P,  z) 
Gi{x,q,p,  z) 
hi{x,q,p,  z) 


R”  X  R  X  R  X  R, 

R”  X  (-00, 1)  X  R  X  R, 
[/(x,2),  r{x,z),  0,  0], 
(x,0,p  +  l,i^(x,z,p),l), 
1  -g. 


□ 

Notice  the  last  proof  shows  that  in  the  BGM  model,  setting  parameters  on  hitting 
switching  boundaries  can  be  implemented  with  the  transition  functions.  Note  also  that 
unlike  the  first  two  proofs,  the  last  construction  uses  a  different  (but  equivalent)  state  space 
for  BGM  and  BAUT.  In  any  case,  we  do  not  use  the  fact  that  BGM  contains  BAUT  in 
further  results.  Also,  we  do  not  compare  among  BGM,  NKSD,  and  BD  here.  We  also  notice 
that  WAUT  and  ASLAUT  contain  TDA. 

Summarizing  results  needed  later,  the  WHS,  BGM,  NKSD,  and  ASL  models  contain  the 
TDA  model;  BD  contains  BAUT.  Thus  in  the  sequel,  when  examining  capabilities  of  these 
models,  the  presentation  concentrates  on  the  TDA  and  BAUT  models,  since  all  capabilities 
possessed  by  both  of  them  are  automatically  possessed  by  all  those  reviewed  above.  Extra 
capabilities  of  the  BGM  model  are  noted  as  warrants. 


§3.11  NOTES 

Our  classification  of  hybrid  phenomena  and  the  review  of  all  but  Witsenhai^en’s  model 
appeared  in  [32].  The  comparison  results  for  those  five  minus  the  ASL  model  is  firom  [31]. 


Chapter  4 

Classification  of  Hybrid  Systems 


In  this  chapter  we  classify  hybrid  systems  according  to  their  structure  and  the  phenomena 
that  they  exhibit.  The  hierarchy  of  classes  we  explore  are  as  follows  (for  both  autonomous 
and  controlled  systems).  First,  there  are  general  hybrid  dynamical  systems  {GHDS).  These 
are  then  refined  to  the  concept  of  hybrid  dynamical  system,  or  simply  hybrid  system,  studied 
in  this  thesis.  Then  there  are  two  restrictions  of  hybrid  systems,  in  which  the  discrete 
dynamics  are  suppressed  and  no  continuous-state  jumps  are  allowed;  switched  systems  and 
continuous  switched  systems. 

Along  the  way  we  also  explicitly  define  the  dynamics  of  general  hybrid  dynamical  sys¬ 
tems. 

§4.1  GENERAL  HYBRID  DYNAMICAL  SYSTEMS 

Recall  the  definition  of  a  general  dynamical  system  (GHDS): 

H  =  [Q,-S,A,G], 

with  its  constituent  parts  defined  in  §1.4.  Also  recall  the  notation  and  shorthand  from  that 
section.  We  now  offer  the  following  observations,  expanded  from  that  section. 

Dynamical  Systems.  First,  note  that  in  the  case  |Q|  =  1  and  A  =  0  we  recover  all  dynam¬ 
ical  systems.  Thus,  the  class  of  GHDS  includes  systems  described  by  ordinary  differential 
equations,  difference  equations,  and  digital  automata  (which  include  finite  automata,  push¬ 
down  automata,  Turing  machines,  Petri  nets,  etc.) 

Hybrid  Systems.  Next,  the  case  |Q1  finite,  each  Xg  a  subset  of  M" ,  and  each  F,  =  E  largely 
corresponds  to  the  usual  notion  of  a  hybrid  system,  viz.  a  coupling  of  finite  automata 
and  differential  equations  [31,  32,  66].  The  two  are  coupled  at  “event  times”  when  the 
continuous  state  hits  certain  boundaries,  prescribed  by  the  sets  Ag.  We  thus  include  the 
examples  containing  the  discrete  phenomena  discussed  in  §3.2:  collisions,  relays,  hysteresis, 
etc.  This  is  proven  mor6  formally  in  §5,  where  we  examine  the  following  useful  extension, 
repeated  from  §1.4: 

A  hybrid  system  is  a  general  hybrid  dynamical  system  with  Q  countable,  and 
with  r,  =  E  (or  E+)  and  X,  C  E^«,  d,  €  E+,  for  all  g  €  Q.  In  the  notation 
above,  it  may  be  written  as 

[Q>  [{•^9}9€Q)  )  {/glgeQl )  -^5  > 
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where  fq  is  a  vector  field  on  Xq  C  • 

Here,  we  may  take  the  view  that  the  system  evolves  on  the  state  space  R*  x  Q,  where  R 
denotes  the  set  of  finite,  but  variable-length  real-valued  vectors.  For  example,  Q  may  be 
the  set  of  labels  of  a  computer  program  and  i  €  R*  the  values  of  all  currently-allocated 
variables.  If  iV  =  sup{dq}qeQ  is  finite,  we  may  take  the  state  to  belong  to  the  so-called 
carrier  manifold  R^ . 

EXAMPLE.  An  interesting  example  motivated  above  is  scientific  calculations.  More  precisely, 
we  can  model  Smale’s  tame  machines  [19]  in  this  framework. 


Changing  State  Space.  The  state  space  may  change.  This  is  useful  in  modeling  com¬ 
ponent  failures  or  changes  in  dynamical  description  based  on  autonomous— and  later, 
controlled— -events  which  change  it.  Examples  include  the  collision  of  two  inelastic  par¬ 
ticles  or  an  aircraft  mode  transition  that  changes  variables  to  be  controlled  [103]. 

We  allow  the  Xq  to  overlap  and  the  inclusion  of  multiple  copies  of  the  same  space.  This 
may  be  used,  for  example,  to  take  into  account  overlapping  local  coordinate  systems  on  a 
manifold  [9]. 

Refinements.  We  may  refine  the  concept  of  general  hybrid  dynamical  system  H  by  adding; 

•  inputs,  including  control  inputs,  disturbances,  or  parameters  (see  general  controlled 
hybrid  dynamical  system  below). 

•  outputs,  including  state-output  for  each  constituent  system  as  for  dynamical  systems 
[§2.1.2]  and  edge-output: 

ff=[Q,E,A,G,0,r?], 

where  tj  :  A O  produces  an  output  at  each  jump  time. 

•  I,F  C  S,  sets  of  initial  or  final  states. 

•  A  :  A  — t  R+ ,  the  jump  delay  map,  which  can  be  used  to  account  for  the  time  which 
abstracted-away,  lower-level  transition  dynamics  actually  take. 

EXAMPLE.  Think  of  modeling  the  closure  time  of  a  discretely-controlled  hydraulic 
valve  or  trade  mechanism  imperfections  in  economic  markets. 


•  Marked  states,  timing,  or  input  and  output  for  any  constituent  system. 

Example  4.1  (Reconciling  Time  Scales)  Suppose  that  each  constituent  dynamical  sys¬ 
tem  T,q  of  H  is  equipped  with  a  timing  map.  That  is  t  =  {TjjgeQ  ivhere 

Tq  :  Xq  xTq  ^  K-J-  • 

Then,  we  may  construct  trajectories  for  H,  i.e.,  a  function  from  “real-time”  to  state. 
This  is  discussed  in  more  generality  in  §4.3  below. 
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Hierarchies.  We  may  iteratively  combine  hybrid  systems  Hq  in  the  same  manner,  yielding 
a  powerful  model  for  describing  the  behavior  of  hierarchical  systems  (cf.  Baas’s  hyper¬ 
structures  [8]  and  Harel’s  statecharts  [69]). 

Adding  Control.  Likewise,  a  general  controlled  hybrid  dynamical  system  (GCHDS)  is  a 
system 

Hc  =  [Q,S,A,G,V,C,F], 

with  its  constituent  parts  defined  in  §1.4. 

The  admissible  control  actions  available  are 

•  the  continous  controls  u  G  Uq,  exercised  in  each  constituent  regime, 

•  the  discrete  controls  v  €  Vq,  exercised  at  the  autonomous  jump  times  (which  occur 
on  hitting  the  set  A), 

•  the  intervention  times  and  destinations  of  the  controlled  jumps. 


NOTES. 

1.  Disturbances  and  other  nondeterminism  may  be  modeled  by  partitioning  U,V,  and 
C  into  portions  that  are  under  the  influence  of  the  controller  or  nature  respectively. 

2.  The  model  includes  that  posed  by  Branicky,  Borkar,  and  Mitter  [32]  and  thus  several 
other  previously  posed  hybrid  systems  models  [5,  9,  38,  114,  135,  152,  §3.1].  It  also 
includes  systems  with  impulse  effect  [10,  §2.2.3]  and  hybrid. automata  [57,  §2.2.6].  See 
§5.2. 


3. 


We  could,  but  do  not  for  pedagogical  reasons  and  later  results,  combine  autonomous 
and  controlled  jumps  by  defining  a  set- valued  aufonomoits  jump  map  G'  :  AuC  — f  2^ 


by 


G{s,V), 

F(s)u{s}, 


s€A, 
s  €  C* 


§4.2  CLASSIFYING  GHDS 

The  scope  of  hybrid  dynamical  systems  presents  a  myriad  of  modeling  choices.  In  this 
section,  we  classify  them  according  to  their  structure  and  the  discrete  phenomena  they 
possess.  Below,  the  prefixes  “c-,”  “d-,”  and  “t-”  are  used  as  abbreviations  for  “continuous-,” 
“discrete-,”  and  “time-”  respectively.  If  no  prefix  is  given,  either  can  be  used. 

NOTE.  To  respect  the  historical  development  of  the  subject  of  hybrid  systems  and  the 
cases  of  current  high  interest,  we  colloquially  use  “discrete”  when  referring  to  the  index 
set,  “continuous”  when  referring  to  the  constituent  state  spaces,  and  “time”  when  referring 
to  the  constituent  transition  semigroups.  We  use  these  even  though  our  model  allows,  for 
instance,  the  index  state  to  be  a  continuum  and  the  constituent  state  spaces  to  be  discrete 
topological  spaces. 

Our  structural  classification  is  roughly  captured  by  the  following  list. 

•  Time-uniform.  The  semigroups  may  be  all  be  the  same  for  each  q,  denoted  S  = 
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•  Continuous- time,  discrete- time,  sampled-data.  Each  constituent  dynamical  sys¬ 
tem  may  be  of  a  special  type  that  evolves  in  continuous-time  (F  =  M),  discrete-time 
(r  =  Z),  or  a  mixture.  However,  if  the  GHDS  is  time-uniform,  we  refer  to  it  by  the 
appropriate  label,  e.g.,  continuous-time- uniform. 

•  C-uniform.  The  ambient  state  space  may  be  the  same  for  each  q,  that  is, 

•  C-Euclidean,  c-manifold.  Each  ambient  state  space  may  be  Euclidean  (subset  of 
R”  in  the  usual  topology)  or  a  smooth  manifold. 

•  D-compact,  d-countable,  d-finite,  n-state.  Special  cases  arise  when  the  index 
space  is  compact,  finite,  or  countably  infinite.  If  |Q|  =  n,  we  say  the  GHDS  is  n-state. 

•  D-concurrent  versus  d-serial.  We  may  or  may  not  allow  more  than  one  discrete 
jump  to  occur  at  a  given  moment  of  time.  This  is  related  to  the  next  classification. 

•  Dynamically- uniform.  The  dynamics  may  be  the  same  for  each  q.  Strictly,  such  a 
case  would  also  require  that  the  system  be  c-uniform  and  time-uniform: 

S  =  [{X,heQ.  {r  </>]  (or  E  =  [{X,},eQ,  {r  /]),  for  all  q  €  Q. 

In  these  systems,  then,  the  interesting  dynamics  arises  from  the  transition  map  Gg. 
Such  is  the  case  with  timed  automata  (see  §2.2.6  for  references). 

•  Deterministic  versus  nondeterministic. 

•  Nonautonomous  versus  autonomous.  The  continuous  (or  discrete)  portions  of  the 
dynamics  may  or  may  not  depend  on  time  (or  count  of  events)  or  external  controls.  To 
distinguish  between  time  and  control  dependence,  we  use  the  next  two  classifications. 

•  Time- varying  versus  time- invariant. 

NOTE.  We  do  not  explicitly  deal  with  time-varying  systems  here,  assuming  it  is  taken 
care  of  in  the  usual  way,  viz.  appending  another  state  to  represent  time. 


•  Controlled  versus  uncontrolled. 

Finally,  a  hybrid  dynamical  system  may  also  be  classified  according  to  the  discrete 
dynamic  phenomena  that  it  exhibits  as  follows  (cf.  [32]). 

•  Autonomous-switching.  The  autonomous  jump  map  G  =  u  is  the  identity  m  its 
continuous  component,  i.e.,  v  :  A->  S  has  i/(x,  q)  =  (x,  q'). 

•  Autonomous-impulse.  G  =  J  is  the  identity  in  its  discrete  component. 

•  Cont rolled-switching.  The  controlled  jump  map  F  is  the  identity  in  its  continuous 
component,  i.e.,  F{x,q)  C  {x}  x  Q. 

•  Controlled-impulse.  F  is  the  identity  in  its  discrete  component. 
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With  this  notation,  our  GHDS  model  admits  some  special  cases: 

•  H  autonomous-impulse  with  \Q\  =  1  and  F  =  M  is  an  autonomous  system  with 
impulse  effect  [10]. 

•  H  c-uniform,  time-uniform,  and  autonomous-switching  is  an  autonomous  switched 
system  [30]. 

•  H  continuous-time-uniform,  c-Euclidean-uniform,  d-countable,  is  what  we  call  a  hy¬ 
brid  system. 

§4.3  GHDS  DYNAMICS 

In  this  section,  we  place  some  restrictions  on  GHDS  in  order  to  prove  some  behavioral 
properties.  We  assume  that  F  is  an  ordered  set  with  the  least  upper  bound  property, 
equipped  with  the  order  topology.  Note  that  this  implies  F  is  a  lattice  [88].  We  also  assume 
addition  to  be  order-preserving  in  the  sense  that  if  a  >  0,  then  a  -I-  6  >  b.  This  last 
assumption  ensures,  among  other  things,  that 

F+  =  {a  €  F  I  a  >  0} 

is  a  semigroup;  likewise  for  F“,  defined  symmetrically.  For  brevity,  we  call  such  a  group 
(semigroup)  time-like. 

EXAMPLE.  The  most  widely  used  time-like  groups  are  M  and  Z  under  addition,  each  in  the 
usual  order.  The  rationals  are  not  allowed  since  they  do  not  have  the  least  upper  bound 
property  (though  they  are  a  lattice).  However,  pZ,  p  €  K,  is  allowed.  Example  semigroups 
are  1L|. ,  pZ+,  and  the  free  monoid  generated  by  a  finite,  ordered  alphabet  (in  the  dictionary 
order). 

In  this  section  we  consider  several  initial  value  problems  for  GHDS.  First,  in  the  time¬ 
like  case,  given  dynamical  system  [X,  F,  ^],  we  may  define  the  positive  orbit  of  the  point 
X  as  P{x)  =  </)(x,  F+). 

NOTE.  The  negative  orbit  B{x)  may  be  defined  even  in  the  non-reversible  case  by  y  €  B{x) 
if  and  only  if  x  6  P{y)  [17]. 


Problem  4.2  (Reachability  Problem)  Compute  the  positive  orbit  for  general  hybrid  dy¬ 
namical  system  H. 

The  positive  orbit  is  the  set  defined  as  follows.  We  consider  only  initial  points  in  /  = 
X\A.  We  restrict  ourselves  to  the  case  where  Aq  is  closed^  and  D,  n  A,  =  0.  Suppose 
So  =  (®05  9o)  €  I-  If  Pqoi^o)  =  <^90 (®o 5 !’''■)  does  not  intersect  Ago  we  are  done:  the  positive 
orbit  is  just  (xo).  Else,  let 

gi  =  inf {5  €  F+  |  (l)qo{xo,g)  €  A^J.  (4.1) 


^We  may  relax  this  to  d*Ag^  defined  in  Equation  (2.4),  trivially. 
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Since  Ago  closed,  T+  is  time-like,  and  (p  is  continuous,  the  set  in  Equation  (4.1)  is  closed, 
gi  exists,  and  €  Aq^.  Define  5i  =  <^90(^1 )  ^  ^  condnue.  D 

When  a  GHDS  is  time-uniform  with  time-like  group  T,  it  induces  a  r+-transition  system 
In  this  case,  we  may  define  its  (forward)  trajectory  as  a  function  from  F  into 

/. 

Problem  4.3  (Trajectory  Problem)  Compute  the  trajectories  for  general  hybrid  dynam¬ 
ical  system  H. 

We  want  to  compute  $(soi9/)  for  9f  ^  ^rid  sq  =  (zo,9o)  €  7.  Let  50  —  0-  The 
construction  and  arguments  are  similar  to  above,  so  we  give  the  {i  —  l)th  step; 

gi+i  =  iTif{g  e  rf  \  g  <  9f  —  9u  4>qi{^ii9)  ^ 

If  the  set  is  ever  empty,  we  are  done.  On  the  interval  g  €  [^ij^i+i)  the  system  evolves 
according  to  (pix^g-  gi).  Also,  gi+i  -gi>0is  assured  by  the  fact  that  topological  grou^ 

are  regular  [113,  p.  145].  j  ^  i  l  i-.  u 

The  above  construction  allows  us  to  formulate  stability  and  fimte-hme  reachability  prob- 

lems.  1  j  j 

Note  that  trajectories  may  not  be  extendible  to  all  of  T  ,  i.e.,  we  have  not  precluded 

the  accumulation  of  an  infinite  number  of  jumps  in  finite  time.  See  Example  10.7.  This 
can  be  removed  in  the  case  of  hybrid  systems  by,  for  example,  assuming  uniform  Lipschitz 
continuity  of  the  vector  fields  and  uniform  separation  of  the  jump  and  destination  set  A 

and  D.  See  §10. 

NOTE.  From  above,  if  =  0  and  A,  closed  then  (positive)  orbits  and  trajectories  exist 

(up  to  a  possible  accumulation  time  of  finite  jumps)  and  are  unique.  Using  [17,  Theorem 
3.4.11],  it  is  enough  that  A,  =  D,  U  (Xg\A,)  be  locally  compact  at  each  z  €  £>,  and  f{x) 
subtangential  to  A,  for  all  i  6  D,  n  dDq. 

Similarly  to  reachability,  we  have  the  following. 

Problem  4.4  (Accessiblity  Problem)  Compute  the  set  of  points  accessible  under  all  ad¬ 
missible  control  actions  from  initial  set  I  for  general  controlled  hybrid  dynamical  system 
system  He* 

The  answer  is  largely  the  same  as  above  except  that  we  must  vary  over  all  admissible 
control  actions. 

§4.4  HYBRID  DYNAMICAL  SYSTEMS 

In  this  section,  we  give.  expUcit  representations  of  the  different  classes  of  hybrid  systems 
arising  firom  the  definitions  above.  We  concentrate  on  the  c-continuous-time,  c-uniform, 
d-finite,  time-invariant,  autonomous  case.  Extensions  to  other  cases  above  are  straightfor¬ 
ward. 


NOTE.  Recall  the  notation  q'^,  introduced  in  §3.2.5. 
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A  (continuous-time)  autonomous-switching  hybrid  system  may  be  defined  as  fol- 


x{t)  =  f{x{t),q{t)), 
q+{t)  =  u{x{t),q{t)), 


(4.3) 


where  x{t)  €  R",  qit)  €  Q  Here,  f{-,q)  :  R"  ^  R",  g  €  Q,  each  globally 

Lipschitz  continuous,  is  the  continuous  dynamics  of  Equation  (4.3);  and  i/  :  R"  x  Q  ->  Q 
is  the  finite  dynamics  of  Equation  (4.3). 

Thus,  starting  at  [xci],  the  continuous  state  trajectory  x(-)  evolves  according  to  x  = 
f{x,i).  If  x(-)  hits  some  at  time  ti,  then  the  state  becomes  [x(ti),  j],  from 

which  the  process  continues. 

Clearly,  this  is  an  instantiation  of  autonomous  switching.  Switchings  that  are  a  fixed 
function  of  time  may  be  taken  care  of  by  adding  another  state  dimension,  as  usual.  Examples 


are  the  Tavernini  and  autonomous  Witsenhausen  models. 

By  a  c-controlled  autonomous-switching  hybrid  system  we  have  in  mind  a  system 

of  the  form;  ,  ,  ^  ^  ^  ^ 

x{t)  =  f{x{t),q{t),u{t)), 

q+lt)  =  i/{x{t),q{t),u{t)), 


where  everything  is  as  above  except  that  u{t)  €  R"*,  with  /  and  u  modified  appropriately. 
An  example  is  Witsenhausen’s  model. 

An  is  a  system 

®(^)  ~  x{t)  ^  M  14.5) 

x'^(t)  =  J(x(t)),  x{t)  €  M 


where  x(t)  €  R",  and  J  :  R"  R”.  Examples  include  autonomous  systems  with  impulse 
effect. 

Finally,  a  hybrid  system  with  autonomous  switching  and  autonomous  impulses  (i.e.,  the 
full  power  of  autonomous  jumps)  is  just  a  combination  of  those  discussed  above: 


x{t)  =  f{x{t),q{t)), 
x+(i)  =  J(x(t)), 

q+{t)  =  u{x{t),q{t)), 

where  x(t)  €  R"  and  q{t)  €  Q  C  Z.  Examples  include  the  BGM  model  and  hence  all  the 
other  autonomous  models  in  §3. 

Likewise,  we  can  define  discrete-time  autonomous  and  controlled  hybrid  systems  by 
replacing  the  ODEs  above  with  difference  equations.  In  this  case.  Equation  (4.3)  represents 
a  simplified  view  of  some  of  the  models  in  [38].  Also,  adding  controls— both  discrete  and 
continuous— is  straightforward.  Finally,  non-uniform  continuous  state  spaces,  i.e.,  x(<)  € 
Xq(t)i  niay  be  added  with  little  change  in  the  foregoing, 

§4.5  SWITCHED  SYSTEMS 


We  have  in  mind  the  following  model  as  a  prototypical  example  of  a  switched  system: 

x{t)  =  fi{x{t)),  i  e  Q  ~  (4.6) 


where  x{t)  €  R".  We  add  the  following  switching  rules. 
•  Each  fi  is  globally  Lipschitz  continuous. 
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•  The  i's  axe  picked  in  such  a  way  that  there  are  finite  switches  in  finite  time. 

Switched  systems  are  of  “variable  structure”  or  “multi-modal” ;  they  are  a  simple  model 
of  (the  continuous  portion)  of  hybrid  systems.  The  particular  i  at  any  given  time  may  be 
chosen  by  some  “higher  process,”  such  as  a  controller,  computer,  or  human  operator,  in 
which  case  we  say  that  the  system  is  controlled.  It  may  also  be  a  function  of  time  or 
state  or  both,  in  which  case  we  say  that  the  system  is  autonomous.  In  the  latter  c^e, 
we  may  really  just  arrive  at  a  single  (albeit  complicated)  nonlinear,  time-varying  equation. 
However,  one  might  gain  some  leverage  in  the  analysis  of  such  systems  by  considering  them 
to  be  amalgams  of  simpler  systems. 

Models  like  Equation  (4.6)  have  been  studied  for  stability  [60,  119].  However,  those 
papers  were  predominantly  concerned  with  the  case  where  all  the  fi  are  linear.  We  discuss 
the  general  cases  in  §8. 

We  also  discuss  difference  equations 

a:[A:  +  l]  =  /i(x[fc-|-l]),  i  €  Q  {1, •  •  •  ,iV},  (4.7) 

where  x[k]  €  M".  Here,  we  only  add  the  assumption  that  each  fi  is  globally  Lipschitz 
continuous.  Again,  these  equations  can  be  thought  of  as  the  “continuous”  portion  of  the 
dynamics  of  hybrid  systems  combining  difference  equations  and  finite  automata  [38]. 

More  abstractly,  a  general  switched  system  is  a  system 

a  =  [Q,  S,  A,  R] , 

where  all  is  as  defined  as  above,  except  the  operators  which  are  restricted  to  be 
autonomous-switching,  and  the  switching  rules  given  by  R.  We  will  not  be  more  pre¬ 
cise  here,  noting  that  the  cases  of  usual  interest  (i.e.,  when  R  depends  on  the  state)  have 
been  taken  care  of  above.  Nevertheless,  the  abstraction  is  helpful,  as  seen  below,  where  we 
concentrate  on  the  non-general  case. 

Our  abstracting  away  of  the  finite  dynamics  in  studying  switched  systems  above  can 
be  motivated  by  “verification  by  successive  approximation”  [2].  For  instance,  consider  the 
following  set  of  (initial  state-dependent)  switching  sequences; 

(C)  Constrained  (hybrid  systems) 

(A)  Arbitrary  (IFS,  see  §8) 

Then,  examining  the  reachability  sets  of  these  systems  with  respect  to  safety  and  liveness 
constraints  given  by  sets,  we  have  the  following  picture: 

•  Safety:  Tl{C)  C  72.(A)  C  Safe 

•  Liveness:  Live  C  7i(C')  C  72.(A) 

Such  containment  relations  allow  one  to  prove  properties  of  the  hybrid  system  by  comparison 
with  the  switched  system  and  vice  versa. 

§4.6  CONTINUOUS  SWITCHED  SYSTEMS 

We  also  study  continuous  switching  systems.  A  continuous  switching  system  is  a 
switching  system  with  the  additional  constraint  that  the  switched  subsystems  agree  at 
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the  switching  time.  More  specifically,  consider  Equation  (4.6)  and  suppose  that  at  times 
tj,  j  =  1,2,3,...,  there  is  a  switch  from  fkj_i  to  fky  Then  we  require  fkj-i{x{tj),tj)  - 
fk  That  is,  we  require  that  the  vector  field  is  continuous  over  time. 

This  constraint  leads  to  a  simpler  class  of  systems  to  consider.  At  the  same  time, 
it  is  not  overly  restrictive  since  many  switching  systems  naturally  satisfy  this  constraint. 
Indeed  they  may  even  arise  from  the  discontinuous  logic  present  in  hybrid  systems.  For 
example,  we  might  have  an  aircraft  where  some  surface,  say  the  elevator,  controls  the 
motion.  But  this  elevator  is  in  turn  a  controlled  surface,  whose  desired  action  is  chosen 
by  a  digital  computer  that  makes  some  logical  decisions.  Based  on  these  decisions,  the 
computer  changes  elevator  inputs  (say  current  to  its  motor)  in  an  effectively  discontinuous 
manner.  However,  the  elevator  angle  and  angular  velocity  do  not  change  discontinuously. 
Thus,  from  the  aircraft’s  point  of  view  (namely,  at  the  level  of  dynamics  relevant  to  it), 
there  are  continuous  switchings  among  regimes  of  elevator  behavior.  Therefore,  continuous 
switching  systems  arise  naturally  from  abstract  hybrid  systems  acting  on  real  objects. 

Another  problem  arises  in  examples  like  the  one  we  just  introduced;  the  problem  of 
unmodeled  dynamics.  Suppose  the  pilot,  from  some  quiescent  operating  point,  decides  to 
invoke  hard  upward  normal  acceleration.  The  elevator  starts  swinging  upward  until  it  is 
swinging  upward  at  maximum  angular  velocity  (in  an  effort  track  the  violent  maneuver 
requested  by  the  pilot).  Then,  some  higher  process  makes  a  calculation  and  decides  that 
continuing  this  command  would  result  in  an  unsafe  configuration  (say  attack  angle  beyond 
critical).  It  decides  to  begin  braking  the  elevator  motor  immediately  to  avoid  this  situation. 
In  this  case,  the  desired  angular  velocity  profile  of  the  elevator  (over  the  whole  move)  is 
most  probably  trapezoidal.  However,  the  elevator  is  a  dynamic  element  that  can’t  track 
that  desired  profile  exactly.  We  may  want  to  know  how  taking  these  unmodeled  dynamics 
into  account  affects  our  already  modeled  dynamics.  We  may  also  want  to  know  how  high 
our  control  gains  should  be  to  track  within  a  certain  error.  In  §8  we  develop  theory  that 
allows  to  answer  both  these  questions. 

§4.7  NOTES 

Our  identification  of  discrete  phenomenon  and  examples  is  from  [32].  We  started  to  classify 
hybrid  systems  in  [30].  That  paper  also  studied  switched  systems.  Continuous  switched 
systems  were  pursued  in  [21],  summarized  in  [28] 
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Chapter  5 

Unified  Hybrid  Systems  Model 


In  this  chapter  we  introduce  an  abstract,  unified  hybrid  systems  model  which  is  shown  to 
capture  all  identified  discrete  phenomena  arising  in  hybrid  systems  and  subsume  all  reviewed 
and  classified  hybrid  systems  models.  The  resulting  model  is  useful  for  posing  and  solving 
hybrid  analysis  and  control  problems. 

§5.1  OUR  UNIFIED  MODEL 

We  now  present  our  over-riding  hybrid  systems  framework  in  generality.  We  refine  it  later 
when  we  set  up  our  control  problem  in  §10.  In  the  nomenclature  of  §4,  the  result  is  a 
controlled  hybrid  system  with  delay  maps.  For  simplicity,  though,  the  destination  sets  are 
specified  a  priori  instead  of  by  the  collection  of  set- valued  maps  F .  Also,  with  no  real  loss 
of  generality,  we  consider  Ui  =  U,Vi  =  V,i  E  Z+.  However,  we  do  generalize  to  allow  the 
vector  field  to  depend  on  the  continuous  state  at  the  last  jump  time. 

Specifically,  our  discrete  state  space  is  Q  =  Z+.  The  continuous  state  space  for  x{-)  is 
X  =  where  each  Xi  is  a  subset  of  some  Euclidean  space  di  E  Z+.  We  also 

specify  a  priori  regions  Ai,CuDi  C  Xu  i  E  Z+.  These  are  the  autonomous  jump  sets, 
controlled  jump  sets,  and  jump  destination  sets,  respectively.  Let  A,  C,  and  D  denote 
the  unions  Ui^t  x  {0.  U  x  {i},  and  Ui  A  x  {i},  i  E  Z+,  respectively.  Let  17,  F  be  the 
sets  of  continuous  and  discrete  controls,  respectively.  The  following  maps  are  assumed  to 
be  known: 

1.  vector  fields  fiiXixXi'x.U  — >  E"** ,  i  E  Z+. 

2.  jump  transition  maps  Gi :  Ai  x  V  D. 

3.  autonomous  transition  delay  Ao,i  :  AixV  -4  E+ . 

4.  controlled  transition  delay  Ac,i  :  A  x  17  — ^  E+ . 

As  shorthand,  we  may  define  G  :  A  x  F  -»•  D  in  the  obvious  manner.  Similarly,  for  Aa  and 

The  dynamics  of  the  control  system  can  now  be  described  as  follows.  There  is  a  sequence 
of  pre-jump  times  {xj}  and  another  sequence  of  post-jump  times  {Fi}  satisfying  0  = 
Fo  <  <  Fi  <  T2  <  F2  <  •  •  •  <  00,  such  that  on  each  interval  [Fj_i,Tj)  with  non-empty 

interior,  a:(‘)  evolves  according  to  the  differential  equation 

x{t)  =  ^(<),  t>0 
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in  some  Xi,  i  €  At  the  next  pre-jump  time  (say,  Tj)  it  jumps  to  some  Dk  €  Xk 
according  to  one  of  the  following  two  possibilities: 


1.  x{Tj)  €  At,  in  which  case  it  must  jump  to  x{Tj)  —  Gi{x{Tj),Vj)  €  D  at  time  Fj  — 
Tj  +  Aaj(x(Tj),Vj),  Vj  €  V  being  a  control  input.  We  call  this  phenomenon  an 
autonomous  jump. 


2.  x{Tj)  €  Ci  and  the  controller  chooses  to — it  does  not  have  to — move  the  trajectory 
discontinuously  to  a;(rj)  €  D  at  time  Fj  =  tj  +  Acj{x{Tj),x{Tj)).  We  call  this  a 
controlled  (or  impulsive)  jump. 


See  Figure  5-1. 


Figure  5-1:  Example  dynamics  of  our  unified  model. 


For  t  €  [0,oo),  let  [t]  =  maxj{Fj  |  Fj  <  t}.  The  vector  field  ^(t)  of  Equation  (3.1)  is 
given  by 

^{t)  =  fi{x{t),x[t],u{t)),  (5.1) 

where  i  is  such  that  x(t),a:[t]  €  Xi  and  u(-)  is  a  17- valued  control  process. 


NOTE.  The  autonomous  version  of  this  model  (including  no  controlled  jumps)  yields  unique 
trajectories  in  the  case  of,  for  instance.  A,  closed  and  Aj  n  Pi  =  0.  See  §4. 
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§5.2  INCLUSION  OF  DISCRETE  PHENOMENA  AND  PREVIOUS 
MODELS 

We  now  show  how  this  framework  encompasses  the  discrete  phenomenon  of  §3.2,  and  how 
it  subsumes  the  hybrid  systems  models  reviewed  in  §3  and  classified  in  §4. 

First,  a  simplification.  If  a  set  of  parameters  or  controls  is  countable  and  discrete,  such 
as  a  set  of  strings,  we  may  take  it  to  be  isomorphic  with  a  subset  of  Z+.  On  the  other 
hand,  consider  a  set  of  parameters  or  controls,  U,  where  U  is,  &  compact,  connected,  locally 
connected  metric  space  U.  By  the  Hahn-Mazurkiewicz  theorem  [75],  U  is  the  continuous 
image  of  [0, 1]  under  some  map  and  thus  we  may  set  U  =  [0, 1]  without  any  loss  of  generality. 
Thus,  we  may  assume  below  without  any  loss  of  generality  that  parameters  and  controls 
take  values  in  a  subset  P  C  . 

Autonomous  Switching.  We  show  that  autonomous  switching  can  be  viewed  as  a  special 
case  of  autonomous  jumps,  which  are  taken  care  of  next.  Consider  the  differential  equation 
with  parameters 

X  =  f{x,p), 

where  x  e  E”,  p  G  F  C  M”"  closed,  and  /  :  E"  x  F  ^  E"  continuous.  Let,  i/ ;  E"  x  F  F 
be  the  function  governing  autonomous  switching.  For  example,  in  the  Tavernini  model,  i/ 
is  the  “discrete  dynamics.” 

Then,  since  E"  has  the  universal  extension  property  [113,  §A],  we  can  extend  /  to  a 
continuous  function  F  :  E"  x  E"*  -t  E".  Now,  consider  the  ODE  on  E"''’’”: 

X  =  F{x,0, 

i  =  0, 

where  x  €  E”,  ^  G  E"*,  and  F  :  E"  x  E*^  ->  E"  continuous.  Let,  the  transition  function  be 
G  :  E"  X  F  — >■  E”  X  F  with  G{x,p)  =  (x,  i/(x,p)). 

Autonomous  Jumps.  This  is  clearly  taken  care  of  with  the  sets  A,. 

Controlled  Switching.  A  system  with  controlled  switching  is  described  by 

i(<)  = /(®(*)»^(^))5  x(0)  =  xo€E‘^, 

where  u(-)  is  a  piecewise  constant  function  taking  values  in  G  C  E"*  and  /  :  E^  x  17  E^ 
is  a  map  with  sufficient  regularity.  There  is  a  strictly  positive  cost  associated  with  the 
switchings  of  u(-).  In  our  framework,  let  x'(*)  =  [x(-),  be  the  new  state  process  with 

dynamics 

i'W  =  /'(*'(«)).  /'(■)  =  !/(•).  cf. 

taking  values  in  X  =  where  each  Xi  is  a  copy  of  E**  x  J7.  Set  Ci  =  Di  —  Xi,  Ai  —  ^ 

for  i  G  Z+.  Switchings  of  u(-)  now  correspond  to  controlled  jumps  with  the  associated  costs. 

Controlled  Jumps.  This  is  clearly  taken  care  of  with  the  sets  Cj. 

Digital  Automata.  A  variety  of  automata  are  automatically  subsumed  by  inclusion  of  the 
Tavernini,  BGM,  NKSD,  ASL,  and  Brockett  models,  which  is  demonstrated  next.  Inclusion 


82  Unified  Hybrid  Systems  Model 


of  general  digital  automata  follows  by  using  a  countable  symbol  set  I  in,  for  instance,  the 
NKSD  model. 

Tavernini  and  BGM  Models.  Let  primed  symbols  denote  those  in  our  model  with  the 
same  notation  as  those  for  BGM.  It  is  obvious  that  our  model  includes  the  BGM  model  by 
choosing  X'^  =  Xi,  Ai  =  dXi  U  dUi,  Di  =  Ui,  Ci  =  0,  and 

G'j{x)  =  {ni{Gj{x));  MGjix))) 

for  X  e  Uj.  G'  need  not  be  defined  on  dXj\dUj,  but  for  completeness  we  may  define 
_  (a;,  for  X  €  dXj\dUj.  Since  BGM  contains  Tavernini’s  model,  our  model  does 

as  well. 

NKSD  and  ASL  Models.  Our  model  includes  the  ASL  model.  First,  choose  Xi  =  R"  xM^, 
i  e  I.  Then,  note  that  the  sets  AD-\t),  i  €  J,  form  a  partition  of  F.  Define  the  sets 
Mj  =  h-^{AD-'^ij))  and  define 

Ai=  U  Mj. 

Then  define 

fi  =  [/,  0,  0,  0] 

with  dimensions  representing  x,  q,  i,  and  o.  The  model  is  complete  by  specifying 

Gi{x)  =  (x,  u{q,  j),  j,  Tiiq)]  j) 


if  X  e  Mj  C  At.  .  .  J  J 

Inclusion  of  NKSD  is  similar.  However,  since  the  resulting  partitions  depend  on  q  one 

must  use  multiple  copies  of  Xi  and  fi  as  above,  one  for  each  AD„  q  €  Q.  We  must 
also  append  the  state  t  to  the  state  vector  X  (and  use  [t]),  with  the  obvious  differential 
equations/transitions.  Finally,  rj  depends  on  both  q  and  j  in  this  case. 


Brockett’s  Models. 

and  defining 


Our  model  includes  Brockett’s  BD  model  by  choosing  Xi  =  R*"  x  R^ 
/  =  [/,  r,  0,  0,  0] 


with  dimensions  representing  x,q  =  p-\p\,i=  bJ >  =  R"  x  {1}  x R  , 

Di  =  R"  X  {0}  X  Z%,  and  Gi((x,l,i,v,2),v')  =  (x,0,i  +  l,v',i/(x,2,t;);  1).  BB  is  seen  to 
be  included  in  the  same  manner,  but  removing  the  state  dimension  for  2.  It  is  clear  that 
this  can  be  extended  to  include  BDV. 


Setting  Parameters  and  Timers.  A  system  which,  upon  hitting  boundaries,  sets  parame¬ 
ters  from  an  arbitrary  compact  set  P  CW  can  be  modeled  in  our  framework  by  redefining 
X'i  =  Xi  X  RP,  and  V'  =  V  x  P,  and  defining  /•  iXixXiXU  xW  as 

fi{x,p,y,q,u)  =  [fi{x,y,u),  0]^ 

&nd  G' :  A  X  P  xV  X  P  D  X  P  as 

G'{x,p,v,p')  =  [G(x,v),  p']’’. 
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each  for  all  possible  arguments.  Likewise,  one  can  redefine  the  switching  cost  and  delay 
appropriately. 

A  system  which  sets  timers  upon  hitting  boundaries  can  be  modeled  by  a  vector  of  the 
rate  equations  in  Brockett’s  BDV  model  of  hybrid  systems,  which  in  turn  can  be  modeled 
in  our  framework  as  previously  discussed. 

§5.3  EXAMPLE 

Consider  again  the  hysteresis  example  of  Equation  (1.1).  For  specificity,  consider  a  system 
with  control,  namely,  /  =  H{x)  +u.  It  can  be  modeled  as  follows.  The  state  space  is 
X  =  {X-bXi},  with  X-x  =  [-A,oo)  and  Xi  -  (-oo,A].  The  continuous  dynamics  is 
given  by 

/_1  =  u-l, 

fx  =  u  + 1. 

The  discrete  dynamics  is  governed  by  the  autonomous  jump  sets  A_i  and  Ax  and  their 
associated  transitions,  which  are,  respectively, 

(— A,— 1)  t-4  (— A, +1), 

(+A,+1)  (+A,-1). 


§5.4  NOTES 

Our  unified  model  first  appeared  in  [32]. 
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Part  II 


Analysis  of  Hybrid  Systems 


Chapter  6 

Topology  of  Hybrid  Systems 


In  this  chapter,  we  discuss  topological  issues  associated  with  hybrid  systems.  Recall  from 
§1.6  that,  in  general,  hybrid  systems  do  not  give  rise  to  trajectories  that  are  continuous 
in  the  initial  condition.  Therefore,  the  best  one  can  hope  for,  in  general,  is  continuity  of 
each  of  the  constituent  maps.  By  construction  though,  each  extended  transition  map,  </>,, 
is  continuous.  In  this  chapter,  we  discuss  topologies  such  that  the  discrete  component  of 
the  transition  map,  is  continuous  for  each  q.  In  particular,  we  examine  topologies 

for  achieving  continuity  of  maps  from  a  set  of  measurements  of  continuous  dynamics  to  a 
finite  set  of  input  symbols  (AD  map). 

Then  we  look  at  “completing  the  loop”  by  composing  the  AD  map  with  that  from  a 
finite  set  of  output  symbols  back  into  the  control  space  for  the  continuous  dynamics  (DA 
map).  Finding  some  anomalies  in  completing  this  loop,  we  discuss  a  different  view  of 
hybrid  systems  that  can  broach  them  and  is  more  in  line  with  traditional  control  systems. 
The  most  widely  used  fuzzy  control  system  is  related  to  this  different  view  and  does  not 
possess  these  anomalies.  Indeed,  we  show  that  fuzzy  control  leads  to  continuous  maps  (from 
measurements  to  controls)  and  that  all  such  continuous  maps  may  be  implemented  via  fuzzy 
control. 

We  end  by  drawing  connections  to  the  previous  and  next  chapters. 

§6.1  INTRODUCTION 

In  traditional  feedback  control  systems — continuous-time,  discrete-time,  sampled-data — the 
maps  from  output  measurements  to  control  inputs  are  continuous  (in  the  usual  metric- 
based  topologies).  When  dealing  with  hybrid  systems,  however,  one  immediately  runs  into 
problems  with  continuity  using  the  “usual”  topologies.  Whereby  we  begin  . . . 

In  this  chapter,  we  discuss  some  results  relating  to  the  topology  of  hybrid  (mixed  con¬ 
tinuous  and  finite  dynamics)  systems.  We  begin  with  a  model  of  a  hybrid  system  as  shown 
in  Figure  6-1. 

We  are  interested  in  maps  from  the  continuous  plant’s  output  or  measurement  space 
into  a  finite  set  of  symbols.  We  call  these  AD  maps.  We  are  also  interested  in  the  map 
from  this  symbol  space  into  the  control  or  input  space  of  the  continuous  plant  {DA  map). 
In  many  control  applications,  both  the  measiurement  and  control  spaces  are  (connected) 
metric  spaces.  Therefore,  we  keep  oxir  discussion  germane  to  such  assumptions. 

The  chapter  is  organized  as  follows:  In  the  next  section,  we  discuss  AD  maps.  First,  we 
illuminate  the  general  issues.  Then,  we  examine  at  length  an  AD  map  proposed  in  [114]. 
We  verify  that  the  map  is  indeed  continuous,  developing  enough  technical  lemmas  to  easily 
add  the  fact  that  the  symbol  space  topology  they  constructed  is  the  same  as  the  quotient 
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Figure  6-1:  Prototypical  hybrid  system. 


topology  induced  by  their  AD  map. 

In  §6.3  we  discuss  what  happens  if  we  try  to  impose  continuity  from  the  measurement 
to  the  control  spaces.  We  first  illuminate  why  this  is  unreasonable  given  the  fact  that  the 
measurement  and  control  spaces  are  normally  connected  metric  spaces.  We  then  impose  a 
new  topology  on  the  control  space  that  gives  rise  to  continuous  maps. 

In  §6.5  we  introduce  a  different  view  of  hybrid  systems.  This  view  allows  us  to  meaning¬ 
fully  discuss  continuity  of  maps  from  the  measurement  to  control  spaces  without  introducing 
new  topologies.  We  also  show  that  the  most  widely  used  fuzzy  logic  control  structure  is 
related  to  this  form,  and  that  it  indeed  is  a  continuous  map  from  measurements  to  controls. 
It  is  further  demonstrated  that  these  fuzzy  logic  controllers  are  dense  in  the  set  of  such 
continuous  functions. 

We  end  with  a  topological  viewpoint  that  reconciles  the  results  in  this  chapter  to  our 
discussion  of  hybrid  system  trajectories  in  the  preceding  one  and  our  definitions  of  simulation 
in  the  next,  simulation,  our  next  topic.  The  Appendix  to  this  chapter  collects  the  proofs 
of  parts  of  a  technical  lemma.  The  thesis  Appendix  §A  reviews  most  of  the  topological 
concepts  used. 

§6.2  CONTINUOUS  AD  MAPS 
§6.2.1  GENERAL  DISCUSSION 

In  this  section,  we  discuss  continuity  of  maps  from  the  measurement  space  of  the  continuous 
plant  into  the  finite  symbol  space.  Such  continuity  is  desirable  when  implementing  control 
loops,  since  we  want,  roughly,  small  changes  in  measurement  to  lead  to  small  changes  in 
control  action. 

The  basic  problem  we  have  in  going  from  the  continuum,  Y ,  into  a  finite  set  of  symbols, 
J,  is  that  I  usually  comes  equipped  with  the  discrete  topology  and  the  only  continuous  maps 
from  Y  to  I  in  this  case  are  constant  (since  Y  is  connected  and  any  subset  of  I  with  more 
than  one  point  is  not).  Therefore,  we  must  search  for  topologies  on  I  which  are  not  the 
discrete  topology.  At  first,  we  may  be  disheartened  by  the  fact  that  this  also  precludes  all 
Hausdorff,  and  even  Ti  topologies,  from  consideration.  However,  the  topologies  associated 
with  (finite)  observations  are  naturally  To  [141].  Fortunately,  there  do  exist  To  topologies 
other  than  the  discrete  topology  on  any  finite  set  of  more  than  one  point: 
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Example  6.1  Suppose  X  is  a  finite  set  having  n  >  1  elements.  There  exists  a  To  topology 
on  X  that  is  not  the  discrete  topology  {and  hence  neither  T\  nor  Hausdorff). 

Proof.  Take  as  a  beisis  the  following  subsets  of  X'.  0,  {xi}, . . . ,  X.  □ 

Using  this  idea,  a  way  of  getting  around  the  problem  above  is  to  append  the  symbol 
space,  /,  with  a  single  new  symbol,  J..  Then,  we  place  the  following  topology  on  I'  = 

2^,1  U  {±},  where  2^  is  the  power  set  of  1.  This  topology  on  I'  makes  it  homeomorphic 
to  X  in  the  proof  above  (when  X  and  I'  have  the  same  number  of  elements).  Therefore, 
it  is  To,  but  not  Ti.  Now,  we  can  create  continuous  maps  from  a  continuum,  Y ,  into  /'  as 
follows:  Let  Ai  be  N  mutually  disjoint  open  sets  not  covering  Y .  Let  I  =  and 

define  f{Ai)  =  i  and  f{Y  -  Ui^i)  =  Y.  We  claim  /  is  continuous.  It  is  enough  to  check 
the  basis  elements  of  the  topology  on  I',  which  are  the  singleton  sets  of  elements  of  I  plus 
the  set  r  itself.  We  have  =  Ai,  open,  for  each  i  €  I.  Further,  /“^(T)  =  Y,  which  is 

open. 

Another  topology  which  works  is  the  following:  0,  {U  U  {X}  \  V  €  2^},  with  the  Ai 
closed  instead  of  open  (see  §6.3  for  a  use  of  a  topology  like  this).  There  are  presumably 
many  other  choices  one  can  make.  Below  we  examine  at  length  one  espoused  in  [114]. 

§6.2.2  AT>  MAP  OF  NERODE-KOHN 
DEFINITION: 

The  AD  map  is  a  map  from  the  measurement  space,  Y,  into  a  finite  set  of  symbols,  I. 
Nerode  and  Kohn  [114]  create  a  continuous  AD  map  as  follows: 

Algorithm  6.2  (Open  Cover  Topology)  1.  First,  take  any  finite  open  cover  of  the 
measurement  space:  Y  -  U"=i  where  the  Ai  are  open  in  the  given  topology  of  Y . 

2.  Next,  find  the  so-called  small  topology,  Ty,  generated  by  the  subbasis  Ai.  This 
topology  is  finite  (as  we  argue  below)  and  its  open  sets  can  be  enumerated,  say, 
as  D\ , ,  Bp . 

3.  Next,  find  all  the  non-empty  join  irreducibles  in  the  collection  of  the  Bi  (that  is,  all 
non-empty  sets  Bj  such  that  if  Bj  =  Bk^Bi,  then  either  Bj  =  Bk  or  Bj  =  Bi).  Again, 
there  are  a  finite  number  of  such  join  irreducibles,  which  we  denote  Ci, . . . ,  Cn- 

4.  Let  the  set  of  symbols  be  /  =  {1, . . . ,  N}.  Further,  define  the  function  AD{y)  =  i  if 
Ci  is  the  smallest  open  set  containing  y. 

5.  Create  a  topology,  7/,  on  /  as  follows.  For  each  i  €  /,  declare  Di  =  {j  \  Cj  C  Cj}  to 
be  open.  Let  7/  be  the  topology  generated  by  the  Di. 

Here  is  a  simple  example  of  the  construction; 

Example  6.3  Let  our  measurement  space  beY  =  [0, 3]  and  the  open  cover  of  this  measure¬ 
ment  space  be 

Ai  =  [0,2),  A2  =  (1,3]. 

The  small  topology  generated  by  this  subbasis  can  be  enumerated  as  follows:  Bi  —  0,  B2  = 
(1,2),  B3  =  [0,2),  B4  =  (1,3],  B5  =  [0,3].  Next,  we  find  the  non-empty  join  irreducibles: 

Cl  =  [0,2),  •  C2  =  (1,3],  C3  =  (1,2). 
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Thus,  we  let  our  set  of  symbols  be  I  =  {1,2, 3}  and  define  the  function  AD  as  follows: 

AD{y)  =  I 


1,  ye [0,1], 

2,  ye  [2,3], 

3,  ye  (1,2). 


The  open  sets  Di  are  found  to  be 

I»i  =  {l,3},  I>2  =  {2,3},  r>3  =  {3} 

and  the  resulting  topology  on  I,  Ti  is 

0,  {3},  {1,3},  {2,3},  {1,2,3}. 

One  can  readily  check  that  7/  is  Tq  and  that  AD  is  continuous. 


□ 


FILTER  INTERPRETATION: 

Here,  we  give  an  intuitive  interpretation  of  the  Nerode-Kohn  approach  to  hybrid  systems 
as  described  in  [114]  (herein,  N-K)  in  terms  of  bandpass  filters.  Our  discussion  covers  both 

AD  and  DA  maps.  .  • 

The  starting  point  of  the  N-K  approach  is  an  assumption  that  one  can  only  realistically 
distinguish  points  up  to  knowing  the  open  sets  in  which  they  are  contained.  Thus,  one  takes 
small  topologies  on  the  measurement  (a.k.a.  plant  output)  and  control  (a.k.a.  plant  input) 
spaces.  The  open  sets  in  these  topologies  correspond  to  events  that  are  distinguishable  and 
achievable,  respectively.  For  example,  they  represent  measurement  error  or  actuator  error 
(or  equivalence  classes  that  are  adequate  for  the  task  at  hand). 

l\IOTE.  However,  the  theory  developed  from  this  principle  is  destined  to  contradict  itself.  In 
particular,  we  have  seen  that  closed  sets  may  be  distinguished  (these  arise  from  the  partition 
of  the  measurement  space  into  symbol  pre-images,  the  so-called  “essential  parts.”)  More 
provocatively,  we  can  distinguish  single  points  in  the  measurement  space.  Consider  as  a 
representative  example  zero  in  [—1,1].  Then  the  open  sets  [-1,1],  (0,1],  ['1,0)  give  us 

information  to  exactly  deduce  a:  =  0. 

A  good  way  to  think  of  the  open  sets  in  the  small  topology  is  as  notch  filters.  On  the 
input  side,  we  can  pass  our  measurements  through  these  filters.  The  level  of  information 
that  we  glean  is.  Did  it  go  through  the  filter  or  not?  Now,  the  total  information  from 
our  sensors  is  summarized  in  the  string  of  Yes/No  answers.^  (Of  course,  we  also  implicitly 
have  the  filters  themselves,  that  link  these  binary  symbols  with  real  regions  of  measurement 
space.)  By  taking  the  intersection  of  all  filters  which  had  a  Yes  answer,  we  obtain  the  join 
irreducible  firom  which  the  measurement  came.  The  input  symbols  of  the  finite  automaton 
are  simply  “names”  given  to  join  irreducibles.  By  also  taking  into  account  the  No  answers, 
we  obtain  a  partition  of  the  measurement  space  into  what  N-K  call  essential  parts.  See 
Figure  6-2 

Likewise,  on  the  output  side  one  constructs  the  join  irreducibles.  The  output  symbols 
of  the  finite  automata  are  exactly  “names”  given  to  these  join  irreducibles.  Now,  the  finite 

iThat  is,  we  can  basically  do  peak  detection  (now  allowing  nonideal  filters).  Biological  auditory  and 
olfactory  systems  may  work  like  this  [77]. 
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Figure  6-2:  Filter  interpretation  of  the  open  cover  Ai. 


automata  controller  is  simply  a  map  from  input  symbols  to  output  symbols  (modulated 
by  its  internal  state).  To  fix  ideas,  let’s  say  that  the' output  symbol  corresponds  to  join 
irreducible  Kj. 

Again,  we  can  think  of  the  control  space  small  topology  as  a  set  of  notch  filters.  Here,  we 
imagine  some  broadband  source  signal  (which  is  not  exactly  flat)  which  we  use  to  produce 
our  control  in  the  following  way;  Instead  of  choosing  a  single  output  from  the  named  join 
irreducible  deliberately  (normal  AD  conversion),  we  simply  construct  one  in  the  correct 
equivalence  class.  We  do  this  by  using  as  a  control  signal  the  signal  that  results  firom 
passing  our  broadband  source  through  each  of  the  filters  (open  sets)  which  intersect  to  form 
the  join  irreducible  Kj. 

It  is  also  interesting  to  note  that  N-K  seem  to  have  adopted  the  idea  (cf.  Appendix  II  of 
[114])  that  the  finite  automaton  and  small  topologies  are  used  to  construct  approximations 
to  maps  from  the  measurement  to  control  spaces,  the  approximation  (of  a  continuous  control 
law)  necessarily  approaching  that  law  as  the  cover  becomes  finer. 

VERIFICATION  OF  CONTINUITY: 

One  of  the  results  of  [114]  is  the  fact  that  their  AD  map  is  continuous  from  (F,  Ty)  to  (/,  7/). 
Namely,  they  give  (without  proof)  the  following  proposition,  whose  proof  we  provide  for 
completeness: 

Proposition  6.4  (Nerode-Kohn)  AD  -.Y  ^  I  is  continuous. 

We  need  several  technical  lemmas  first,  which  are  also  used  to  prove  later  results: 

Lemma  6.5  The  non-empty  join  irreducibles  of  the  topology  generated  by  the  {subbasis)  Ai 
are  exactly  those  sets  which  can  be  written  as 

v,=  n 

j€J{y) 

where  J{y)  is  the  set  of  all  j  such  that  y  E  Aj. 
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Proof.  Pick  yGY  arbitrarily.  Then  Vy  is  non-empty  since  it  contains  y.  Next,  suppose 
that  Vy  is  not  join  irreducible,  so  that  it  can  be  written  asVy  =  AuB  where  A  ^Vy  and 
B  ^  Vy.  Then  we  must  have  that  y  €  A  or  y  G  B,  or  both.  Without  loss  of  generality, 
assume  y  e  A.  But,  since  A  is  an  element  of  the  topology  generated  by  the  Ai,  it  must  be 
of  the  form  A  -  (Jig/  HjeJi  J,  C  {1, . . . ,  n},  for  each  i  £  /,  that  is,  an  arbitrary 

union  of  finite  intersections  of  elements  of  the  subbasis.  Since  y  €  A,  it  must  be  in  at  least 
one  of  the  sets  in  the  union,  say,  the  kth:  y  €  However,  this  means  y  €  Aj  for 

each  j  €  Jfe.  By  definition,  4  C  J(y),  so  that  Vy  C  A.  But,  since  P,  =  A  U  B,  we  also  haw 

Vy  D  A.  So  that  A  =  Pj,,  a  contradiction.  P 

Thus,  AB  is  a  well-defined  function,  with  AD{y)  =  i  where  Ci  is  the  sma,llest  join 
irreducible  containing  y.  In  fact,  Ci  equals  the  Pj,  defined  in  the  lemma.  Now,  it  is  easy  to 
see  that  the  set  of  non-empty  join  irreducibles  is  finite:  there  are  at  most  2"  -  1  distinct 
non-empty  sets  that  can  be  written  in  this  manner.  Thus,  the  topology  generated  by  the 
Ai  has  less  than  2^"  elements  (since  each  element  is  a  union  of  basis  sets). 

Lemma  6.6  1.  The  non-empty  join  irreducibles  Ci  form  a  basis  of  the  topology,  Ty, 

which  they  generate. 

2.  The  sets  Di  are  a  basis  for  the  topology,  Ti,  which  they  generate. 

3.  AD  is  surjective. 

4.  If  f  is  surjective,  f{f~^{X))  =  X- 

5.  Cj  =  AD-\Dj). 

Proof.  The  detailed  proofs  appear  in  the  Appendix.  Items  1  through  3  are  straightforward. 
(For  those  with  a  knowledge  of  lattice  theory,  the  Ci  and  Di  are  lower  closures  in  their 
respective  lattices  and  give  rise  to  the  (dual)  Alexandrov  mpobgies  thereupon  [141].)  Item 
4  is  in  [113,  p-  20].  Item  5  is  almost  immediate  in  the  D-direction  and  follows  with  the  help 

of  Lemma  6.5  in  the  C-direction.  ■  ^ 

Now,  W6  are  ready  to  prove  the  proposition:  ^ 

Proof,  (of  Prop.  6.4)  Lemma  6.6  says  the  Dj  are  a  basis  and  that  AD  {Dj)  =  Cj, 

which  is  open  in  P.  ^ 

Ti  is  the  quotient  TOPOLOGY: 

Next,  we  want  to  show  that  the  AB  topology  of  Nerode  and  Kohn,  7/,  is  exactly  the  quotient 
topology  of  their  AB  map.  This  is  accomplished  by  proving  that  7/  is  both  coarser  and 
finer  than  the  quotient  topology.  The  following  is  well-known  [113,  p.  143]; 

Let  X  be  a  space;  let  A  be  a  set;  let  p  :  X  A  be  a  surjective  map.  Then  the 
quotient  topology  on  A  induced  by  p  is  the  finest  (i.e.,  largest)  topology  relative 
to  which  p  is  continuous. 

Since  the  AB  map  is  continuous  in  7/  and  surjective,  we  trivially  have:  7/  is  coarser  than 
the  quotient  topology,  Tq,  corresponding  to  AD.  Now,  it  remains  to  show  that  7/  is  finer 
than  Tq.  Proof.  Suppose  J  is  open  in  Tq.  Then  AD~^{J)  is  open  in  Ty.  Finally,  it  can 
be  written  as  AD-^{J)  =  U^€B  ^  of  N},  since  the  Cp  are 
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a  basis  for  Ty.  We  want  to  show  that  J  €  7/.  But  note  that  since  AD  is  surjective 
J  =  AD{AD-\J))  =  AD  f  U  )  =  U  ^D{Cp). 

\peB  ) 

Now,  we  have  from  Lemma  6.6,  that  =  AD-^{Dp).  Since  AD  is  surjective,  this  implies 
AD{C^)  =  AD{AD-'^{D^))  =  D^  So  that  J  =  U/jsb  which  is  open  in  7/,  being  a  union 
of  basis  elements.  CD 

Summarizing,  we  have  shown 

Theorem  6.7  The  AD  topology  of  Nerode  and  Kohn,  Ti,  is  exactly  the  quotient  topology 
of  their  AD  map. 

We  have  also  gotten  something  else  along  the  way.  In  the  last  proof  we  showed  that 
AD{Cp)  =  Dp.  Prom  Lemma  6.6,  we  have  AD~^{Dp)  =  Cp  and  that  Dp  and  Cp  are  bases 
for  77  and  7y ,  resp.  Thus,  AD  is  a  homeomorphism  between  the  topological  spaces  (F,  7y) 
and  (This  homeomorphism  was  also  noted  without  proof  in  [114].) 

§6.3  COMPLETING  THE  LOOP 

§6.3.1  PROBLEMS  COMPLETING  THE  LOOP 

In  this  section,  we  discuss  problems  which  arise  when  considering  continuous  mappings  from 
the  measurement  to  control  spaces  (see  Figure  6-1).  Specifically,  we  have 

Remark  6.8  IfY  is  connected  and  U  is  Ti,  the  only  continuous  maps  from  Y  to  a  finite 
subset  ofU  (i.e.,  /(T)  =  {tti, . . .  ui,...,un  eU)  are  constant  maps. 

Proof.  First,  constant  maps  are  always  continuous,  and  their  image  is  a  single  point  of 
17,  hence  finite.  Next,  suppose  for  contradiction  that  /  is  a  non-constant  continuous  map 
from  Y  into  U  and  the  image  f{Y)  =  {ui, . . . ,  uat},  where  the  Ui  are  distinct  points  in  U 
for  some  finite  N  greater  than  or  equal  to  two.  Since  U  is  Ti,  we  can  construct  open  sets 
V--r  for  i  ^  i  such  that  F  -  contains  Ui  but  not  Uj.  Thus,  there  is  an  open  set  about  ui 

not  containing  U2, _ viz.,  V  =  f]j—2^ij‘  Also,  we  can  construct  an  open  set  which 

contains  each  U2,  ■  •  •  ,^7^  yet  does  not  contain  ui:  W  =  Ui^2  ^,T’  Therefore,  f{Y)  =  VUW 
is  not  connected. 

§6.3.2  TOPOLOGIES  COMPLETING  THE  LOOP 

In  the  previous  subsection,  we  saw  that,  under  mild  assumptions,  there  are  no  non-constant 
continuous  maps  from  the  measurement  to  control  spaces.  In  this  subsection,  we  wish  to  give 
a  topology  on  the  (augmented)  control  space  which  allows  us  to  construct  a  non-constant 
continuous  map. 

We  make  no  assumptions  on  Y  and  U  (except  those  implicit  in  the  definition  of  / 
below).  Suppose  that  the  topology  on  U  is  T.  Then  we  let  U'  =  U  U  {±},  that  is, 
we  append  a  single  element,  J.,  to  C/.  Next,  we  define  a  topology,  "T' ,  on  U'  as  follows: 
7"'  =  0,  {V  U  {i.}  I  V  €  T}.  Suppose  we  wish  to  have  image  points  ui, . . . ,  utf  in  U .  Let 
f-^{ui)  =  Ki  be  disjoint  closed  sets  not  covering  Y.  Let  f{Y  -  Uili  ^i)  =  Then 

Remark  6.9  /  is  continuous. 
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Proof.  /"H®)  =  05  which  is  open.  Now,  suppose  V  is  any  non-empty  open  set  of  U' . 
Then  V'  =  V  \J  {-L},  where  V  is  open  in  U.  Therefore, 

f-\v')  =  r\v)ur\{i}) 

=  y-  U 

iei-J 

(where  J  is  the  set  of  indices  j  for  which  Uj  £V,  and  I  =  {1,  •  •  • ,  N})  which  is  open  since 
its  complement  is  closed;  the  formula  is  well-defined  if  J  ^  J.  If  J  =  /,  then  /  {V  )  —  T, 
which  is  open.  ^ 

§6.4  A  DIFFERENT  VIEW  OF  HYBRID  SYSTEMS 

We  wish  to  propose  a  different  view  of  hybrid  systems  as  shown  in  Figure  6-3. 


Figure  6-3:  Alternative  prototypical  hybrid  system. 

The  difference  between  this  and  the  previous  prototypical  hybrid  system  is  that  there  is 
feedback  on  the  signal  level.  This  feedback  modulates  the  symbols  coming  down  from  the 
higher  level.  Alternatively,  one  can  view  the  symbols  as  specifying  one  of  several  controllers 
whose  output  is  to  be  the  control  signal. 

The  most  widely  used  fuzzy  control  scheme  is  related  to  this  model  in  the  sense  that 
it  achieves  continuous  ipaps — despite  a  finite  number  of  so-called  fuzzy  rules  by  utilizing 
the  continuous  measurement  information.  We  discuss  this  in  more  detail  below. 


§6.4.1  WHY  THE  DIFFERENT  VIEW? 

Before,  we  had  a  natural  fan-in  of  sensory  information  from  the  signal  to  symbol  levels. 
This  models  abstraction  and  reduction.  In  our  new  view,  we  also  have  an  analogous,  natural 
fan-out  of  control  commands  from  the  symbol  to  signal  level  that  was  not  present  before. 
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Basically,  we  are  saying  that  the  finite  description  of  the  plant’s  dynamics  as  seen  from 
automaton’s  point  of  view  is  not  an  exact  aggregation  of  the  plant’s  dynamics.  Therefore, 
one  should  utilize  the  continuous  information  present  at  the  lower  level  as  well  as  the  discrete 
decision  made  above  in  order  to  choose  a  control  input  for  the  lower  level.  There  is  no  need 
to  arbitrarily  pick  a  member  from  the  set  of  controls  (fixed  for  normal  AD  conversion, 
always  arbitrary  in  the  Nerode-Kohn  view).  Instead,  the  set  is  given  by  the  automaton, 
while  the  member  of  that  set  is  chosen  using  information  firom  the  lower  level.  Thus,  the 
aggregated  and  continuous  dynjimics  are  related,  but  the  first  is  not  a  substitute  for  the 
latter.  If  it  were,  the  plant  could  have  been  modeled  directly  as  a  finite  automaton. 


§6.5  EXAMPLE  OF  THE  DIFFERENT  VIEW:  FUZZY  CONTROL 

We  now  wish  to  examine  the  different  view  of  hybrid  systems  as  shown  in  Figure  6-3.  Such 
a  view  can  give  rise  to  a  continuous  map  completing  the  loop.  In  particular,  the  most 
widely  used  fuzzy  control  scheme  is  related  to  this  model  in  the  sense  that  it  achieves  con¬ 
tinuous  maps — despite  a  finite  number  of  so-called  fuzzy  rules — by  utilizing  the  continuous 
measurement  information.  We  discuss  this  in  more  detail  below. 


THE  CONTROL  SCHEME: 

.A  fuzzy  control  scheme  is  given  by  the  commuting  diagram  of  Figure  6-4,  where  F  denotes 
fuzzification,  G  the  inference  map  of  the  fuzzy  rule  base,  and  D  defuzzification.  Here,  the 
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[mai  (x),  •  •  • ,  MAm  (a:) ,  •  •  • ,  f^B'^  (a:)] 
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Figure  6-4:  Fuzzy  Logic  Controller. 


fuzzy  controller  has  M  rules  of  the  form 

RULEi :  IF  X  is  Aj,  THEN  z  is  Bi,  i  6  {1, . . . ,  M} 

and  is  a  cross  product  of  the  space  of  fuzzy  sets  on  Z.  The  most  widely  used  inference 
rule  computes  =  Ta.m{nAilx),  for  all  z  €  Z,  and  defuzzifies  using  the 

centroid:  z  =  where  t  equals  the  centroid  of  hb'.-  = 

ifz  ^  t^B[{z)dz)/{Sz 

The  finite  rule  base  is  related  to  the  finite  symbols  of  our  hybrid  model.  For  instance,  the 
rules  which  fire  are  akin  to  the  filters  which  passed  data  in  our  discussion  of  the  Nerode-Kohn 
approach.  However,  here  one  utilizes  the  underlying  continuous  information  (represented 
in  the  continuous  membership  functions  for  the  measurement  space)  in  order  to  construct 
the  precise  control  output.  Thus,  it  fits  into  our  different  view.  See  Figure  6-5 
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X 


Figure  6-5:  Filter  interpretation  of  the  fuzzy  rule  antecedents  Aj. 


PRODUCING  CONTINUOUS  MAPS: 

We  deal  with  the  prototypical  case  where  X  and  Z  are  closed  intervals  in  R  (for  specificity, 
[a,  6]  and  [c,d],  resp.).  The  case  where  X  is  a  multi-interval  in  R”^  is  a  straightforward 
extension.  The  case  where  z  is  a  multi-interval  in  R"  then  follows  from  considering  each 
dimension  componentwise.  We  claim  that  the  induced  map  g  =  D  o  G  o  F  is  continuous 
from  X  to  Z.  We  assume,  for  the  proof,  that  the  fiAi  and  UBi  are  continuous  on  X  and  Z, 
resp.  This  is  fairly  typical  (e.g.,  triangular  functions). 

Proof.  If  the  maj  are  continuous,  then  F  is  continuous.  It  is  also  easy  to  see  that 
centroid  defuzzification,  D,  is  continuous.  It  remains  to  show  that  G  is  continuous.  Well, 
a  fuzzy  inference  rule  gives  rise  to  the  following  situation:  Haj{z)  =  min{a, /(z)},  where 
Ha  /,  /,  and  a  are  playing  the  role  of  fixed  /x^',  MB;,  and  resp.  Thus,  by  assumption, 

/(z)  €  C([c,  d]  ->•  [0, 1]).  Now,  we  need  G  to  be  continuous  as  a  map  from,  componentwise, 
[0, 1]  to  C([c,  d]  [0, 1]).  But,  if  lai  -  021  <  e,  -  Ha^jW  <  ^  where  H  •  1|  denot^ 

the  sup  norm. 

APPROXIMATING  CONTINUOUS  MAPS: 

Fuzzy  control  maps  are  also  dense  in  the  set  of  continuous  functions  from  X  to  Z.  It 
is  enough  to  note  that  triangular  functions,  which  are  prevalent  for  descriptions  of  fuzzy 
membership  sets,  are  so  dense.  To  more  easily  see  this,  note  that  triangular  functions  can 
be  combined  to  construct  arbitrary  piecewise  linear  functions. 

§6.6  A  UNIFYING  TOPOLOGICAL  VIEWPOINT 

As  open  and  closed  sets  play  dual  roles  in  topology,  we  can  define  topologies  based  on 
closed  sets  by  replacing  “open”  with  “closed”  in  Algorithm  6.2.  Of  course,  one  still  runs 
into  the  problem  that  essential  parts  may  be  points.  However,  we  provide  a  construction 
below  that  avoids  this.  Further,  it  relates  the  topological  results  of  this  chapter  back  to  our 
assumptions  for  the  existence  and  uniqueness  of  hybrid  systems  trajectories  and  to  insuring 
finitely  many  jumps  in  finite  time  [§5].  Further,  it  is  related  to  our  simulation  definitions 

in  §7. 
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For  sake  of  argument,  assume  that  the  measurement  space  is  a  bounded  metric  space. 
Then  the  following  two  constructions  are  dual. 

•  Pick  as  open  cover,  Y  plus  a  finite  number  of  disjoint  open  sets  separated  by  at  least 
some  distance  e. 

•  Pick  as  “closed  cover,”  Y  plus  the  disjoint  closures  of  a  finite  number  of  open  sets. 

In  each  case,  F  -  Ui  is  mapped  to  “continue”  and  the  others  to  different  symbols.  The 
advantages  of  the  latter  construction  are  that:  (1)  each  ADq  map  is  continuous;  (2)  it  is 
consistent  with  the  conditions  for  the  dynamics  to  be  uniquely  defined. 

6.7  NOTES 

The  majority  of  this  chapter  appeared  in  [24].  The  topological  view  of  §6.6  did  not  appear 
but  was  presented  in  the  associated  conference  talk. 

The  filter  interpretation  of  the  Nerode-Kohn  approach  arose  fi:om  discussions  with  Anil 
Nerode. 


6.8  APPENDIX 

This  appendix  collects  the  full  proofs  for  statements  1-3  and  5  in  Lemma  6.6.  They  are 
listed  as  separate  lemmas  for  convenience. 

Lemma  5.1  The  non-empty  join  irreducibles  Ci  form  a  basis  of  the  topology,  Ty,  which 
they  generate. 

Proof.  Each  y  €  F  is  contained  in  such  a  set  since  the  Ai  are  a  cover  of  F .  The  intersection 
of  two  such  sets  that  contain  the  point  y  is  a  superset  of  Vy.  □ 

Lemma  5.2  The  sets  Di  are  a  basis  for  the  topology,  Ti,  which  they  generate. 

Proof.  Each  i  e  I  is  contained  in  Di  since  QcCi,  so  there  is  a  basis  element  containing 
each  i  €  I.  If  f  €  /  belongs  to  the  intersection  of  two  basis  elements,  say  and  Dj^,  theii 
we  need  a  basis  element  Dj^  containing  i  such  that  Dj^  C  Dj^  n  Dj^.  But  then  Ci  C  Cj^ 
and  Ci  C  Cj^.  So  that  Ci  C  Cj^  n  Cj^.  From  this,  we  want  to  show  that  Di  is  contained  in 
Dj^  n  Dj2-  But  this  is  evident  from  the  definition  of  £),: 

Di  =  {j\CjCCi}. 

So,  if  j  €  Di,  then  Cj  C  Ci  C  Cj^,  so  that  j  €  Dji.  Likewise,  Cj  C  Ci  C  Cj^,  so  that 
j  e  Dj2.  Therefore,  Di  C  Dj^  n  Dj^,  is  the  required  basis  element.  □ 

Lemma  5.3  AD  is  surjective. 

Proof.  Pick  i  €  L  By  construction,  there  exists  y  e  Y  such  that  Ci  is  the  smallest 
non-empty  join  irreducible  containing  y.  AD{y)  =  i.  d 
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Lemma  5.5  Cj  =  AD  ^{Dj) 

Proof. 

1.  Cj  D  AD-'^{Dj). 

AD-^{Dj)  =  AD-'^{UkeDjk) 

=  [j  AD-Hk)C  [j  CkCCj. 
k€Dj  keDj 

The  last  inequality  follows  from  the  fact  that  Ck  C  Cj  for  all  A:  €  Dj. 

2.  Cj  C  AD~'^{Dj).  Suppose  y  E  Cj.  Then  either  Cj  is  the  smallest  non-empty  join 
irreducible  containing  y,  in  which  case  we  axe  done,  or  there  is  some  other  smallest 
non-empty  join  irreducible  Ck  containing  y.  We  claim  Ck  C  Cj,  in  which  case  k  €  Dj 
and  y  e  AD~^{Dj),  which  is  the  desired  result. 

Therefore,  it  remains  to  show  that  Ck  C  Cj.  The  smallest  join  irreducible  containing 
y  is  (see  Lemma  6.5)  is  Vy  =  ni€./(y)  ^  y  €  Aj. 

However,  Cj  is  also  a  join  irreducible,  so  that  it  can  be  written  Cj  =  flje j  Aj  for  some 
J  C  (1, . . .  ,n}.  But  Cj  contains  y,  so  that  each  of  the  Aj  in  the  intersection  must 
contain  y.  So  that  by  definition  J  C  J{y),  whence  Ck  =  Vy  C.  Cj. 


□ 


Chapter  7 

Complexity  and  Computation  in 
Hybrid  Systems 


We  explore  the  simulation  and  computational  capabilities  of  hybrid  and  continuous  dynam¬ 
ical  systems.  Notions  of  simulation  of  a  discrete  transition  system  by  a  continuous  one  are 
developed.  We  show  that  hybrid  systems  whose  equations  allow  a  precise  binary  timing 
pulse  (exact  clock)  can  simulate  arbitrary  reversible  discrete  dynamical  systems  defined  on 
closed  subsets  of  R”.  We  also  prove  that  any  discrete  dynamical  system  in  Z"  can  be  sim¬ 
ulated  by  continuous  ODEs  in  M^n+i  We  use  this  to  show  that  there  are  smooth  ODEs 
in  that  possess  the  power  of  universal  computation.  We  use  the  famous  asynchronous 
arbiter  problem  to  distinguish  between  hybrid  and  continuous  dynamical  systems. 


§7.1  INTRODUCTION 

In  this  chapter,  we  explore  the  simulation  and  computational  capabilities  of  hybrid  systems. 
This  chapter  is  a  step  towards  the  characterization  of  these  models  in  terms  of  the  types 
of  systems  that  can  be  described  by,  or  “implemented”  with,  their  equations.  By  construc¬ 
tion,  however,  a  hybrid  system  model  can  implement  ODEs  with  continuous  vector  fields 
(continuous  ODEs).  Thus,  even  with  no  discrete  dynamics,  these  models  can  describe  a 
large  variety  of  phenomena. 

In  addition  to  “implementing”  ODEs,  all  reviewed  models  can  implement  a  precise 
binary  timing  pulse  or  “exact  clock”  (defined  later).  Thus,  we  explore  the  capabilities  of 
systems  with  continuous  ODEs  and  exact  clocks.  For  instance,  we  show  such  systems  can 
simulate  arbitrary  reversible  discrete  dynamical  systems  defined  on  closed  subsets  of  M" . 
These  simulations  require  ODEs  in  which  use  an  exact  clock  as  input. 

Later,  we  find  that  one  can  still  simulate  arbitrary  discrete  dynamical  systems  defined 
on  subsets  of  Z"  without  the  capability  of  implementing  an  exact  clock:  one  can  use  an 
approximation  to  an  exact  clock.  Such  an  “inexact  clock”  is  implemented  with  continuous 
functions  of  the  state  of  a  one-dimensional  continuous  ODE.  As  a  result,  one  can  perform 
such  simulations  using  continuous  ODEs  in  .  Turning  to  computational  abilities,  we 
show  that  continuous  ODEs  in  R^  possess  the  ability  to  simulate  arbitrary  Turing  machines, 
pushdown  automata,  and  finite  automata.  By  simulating  a  universal  Turing  machine,  we 
conclude  that  there  exist  ODEs  in  R^  with  continuous  vector  fields  possessing  the  power  of 
universal  computation.  Further,  the  ODEs  simulating  these  machines  may  be  taken  smooth 
and  do  not  require  the  machines  to  be  reversible  (cf.  [108]). 

Finally,  we  show  that  hybrid  dynamical  systems  are  strictly  more  powerful  than  Lipschitz 
ODEs  in  the  types  of  systems  they  can  implement.  For  this,  we  use  a  nontrivial  example:  the 
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famous  asynchronous  arbiter  problem  [26,  100,  144].  First  we  quickly  review  the  problem. 
Then  we  settle  it  in  an  ODE  framework  by  showing  that  one  cannot  build  an  arbiter  out 
of  devices  modeled  by  Lipschitz  ODEs.  Next,  we  examine  the  problem  in  a  hybrid  systems 
framework.  We  show  that  all  the  hybrid  systems  of  §3  can  implement  an  arbiter  even  if 
their  continuous  dynamics  is  a  system  of  Lipschitz  ODEs. 

The  chapter  is  organized  as  follows.  In  §7.2  notions  of  simulation  are  discussed.  Here,  we 
make  precise  what  we  mean  by  “simulation”  of  discrete  transition  systems  by  continuous 
transition  systems.  All  our  simulation  results  are  collected  in  §7.3.  §7.4  deals  with  the 
asynchronous  arbiter  problem.  The  Appendix  collects  some  technical  lemmas. 

§7.2  NOTIONS  OF  SIMULATION 

In  dynamical  systems,  simulation  is  captured  by  the  notions  of  topological  equivalence  and 
homomorphism  [59,  67,  129].  One  can  extend  these  notions  to  systems  with  inputs  and 
outputs  by  also  allowing  memoryless,  continuous  encoding  of  inputs,  outputs,  and  initial 
conditions. 

In  computer  science,  simulation  is  based  on  the  notion  of  “machines  that  perform  the 
same  computation.”  This  can  be  made  more  precise,  but  is  not  reviewed  here  [20,  104]. 

Other  notions  of  simulation  (for  discrete  dynamical  systems)  appear  in  [91].  All  these 
notions,  however,  are  “homogeneous,”  comparing  continuous  systems  with  continuous  ones 
or  discrete  with  discrete.  One  that  encompasses  simulation  of  a  discrete  transition  system 
by  a  continuous  transition  system  is  required  here. 

One  notion  that  associates  discrete  and  continuous  transition  systems  is  global  section 
[129].  The  set  .Sx  C  A  is  a  global  section  of  the  continuous  dynamical  system  [A,R+,  /] 
if  there  exists  a  to  £  1K+  such  that 

Sx  =  {fiP,kto)  I  A:€Z+}, 

where  P  is  a  set  containing  precisely  one  point  from  each  of  the  trajectories  /  (p,  ®-). ),  p  €  X. 
Using  this  for  guidance,  we  define 

Definition  7.1  (S-simulation)  A  continuous  transition  system  [X,  R). ,  /]  simulates  via 
section  or  S-simulates  a  discrete  transition  system  [Y,Z+,F]  if  there  exist ji  continuous 
surjective  partial  function  xp  :  X  Y  and  to  €  Rf  such  that  for  all  x  E  xp  (Y)  and  all 
k  G.  Z4. 

xp{f{x,kto))  =  F{xp{x),k). 

Note  that  surjectivity  implies  that  for  each  y  €  Y  there  exists  x  €  xp  ^(y)  such  that 
the  equation  holds.  Here,  continuous  partial  function  means  the  map  from  xp-''-{Y)  (as  a 
subspace  of  X)  to  Y  is  continuous. 

Intuitively,  the  set  V  =  xp-^{Y)  may  be  thought  of  as  the  set  of  “valid”  states;  the  set 
A\U  as  the  “don’t  care”  states.  In  dynamical  systems,  V  may  be  a  Poincare  section,  A\U 
the  set  of  points  for  which  the  corresponding  Poincare  map  is  not  defined  [67,  73].  In  com¬ 
puter  science  and  electrical  engineering,  V  may  be  the  set  of  circuit  voltages  corresponding 
to  a  logical  0  or  1;  X\V  the  voltages  for  which  the  logical  output  is  not  defined. 

S-simulation  is  a  strong  notion  of  simulation.  For  instance,  compare  it  with  topological 
equivalence.  Typically,  though,  the  homogeneous  notions  of  simulation  do  not  expect  time 
to  be  parameterized  the  same  (up  to  a  constant)  for  both  systems.  For  example,  a  universal 
Turing  machine,  U,  may  take  several  steps  to  simulate  a  single  step  of  any  given  Turing 


§7.2  Notions  of  Simulation  101 


machine,  M.  Moreover,  the  number  of  such  U  steps  to  simulate  an  M  step  may  change 
from  M  step  to  M  step.  Some  of  the  notions  of  simulation  defined  in  [91]  also  allow  this 
generality.  Further,  the  definition  of  topological  equivalence  of  vector  fields  (different  than 
for  dynamical  systems,  see  [67],)  is  such  that  parameterization  of  time  need  not  be  preserved. 
Thus,  following  the  definitions  in  [91]  one  formulates 

Definition  7.2  (P-simulation)  A  continuous  transition  system  [X,  1R+,/]  simulates  via 
points  or  P-simulates  o  discrete  transition  system  [V,  Z+,  F]  if  there  exists  a  continuous 
surjective  partial  function  tj)  :  X  Y  such  that  for  all  x  €  i{’~^(Y)  there  is  a  sequence  of 
times  0  =  to  <  <  *2  <  •  •  •>  linifc_^oo  tk  =  oo>  such  that 

‘tl’if{x,tk))  =  F{'il;ix),k). 

One  readily  checks  that  S-simulation  implies  P-simulation.  This  is  a  weak  notion.  For 
instance,  consider  the  case  where  Y  is  finite,  |y|  =  N.  Suppose  [X, M-).,/]  has  a  point  p 
such  that  l/(p,R+)|  >  iV  and  p  =  /(p,to)  for  some  to  >  0.  That  is,  the  orbit  at  point  p 
is  periodic  and  contains  more  than  JV  points.  Clearly,  one  may  associate  N  distinct  points 
in  f{p,R+)  with  the  points  in  T,  so  that  [X,l+,/]  P-simulates  [Y,Z+,F].  This  weakness 
persists  even  if  Y  is  infinite.  For  example,  the  simple  harmonic  oscillator  defined  on  the 
imit  circle,  X  =  5^: 


Xi  —  X2 , 

X2  = 

along  with  ip(x)  =  Xi  P-simulates  every  [[-1, 1],Z+,F].  These  arguments  also  show 
the  weakness  of  some  of  the  definitions  in  [91].  Finally,  this  same  example  shows  P- 
simulation  does  not  imply  S-simulation:  the  harmonic  oscillator  above  cannot  S-simulate 
any  [[-1, 1],Z+,F]  for  which  0  is  a  fixed  point  and  1  is  not  a  fixed  point. 

Thus,  P-simulation  need  not  correspond  to  an  intuitive  notion  of  simulation.  The  reason 
is  that  one  wants,  roughly,  homeomorphisms  from  orbits  to  orbits,  not  from  points  to  points. 
As  mentioned  in  §2.1,  this  is  achieved  with  continuous  dynamical  systems.  However,  this 
is  not  possible  with  nontrivial  nonhomogeneous  systems  since  a  discrete  orbit  with  more 
than  one  point  is  a  (countable)  disconnected  set  and  any  non-constant  continuous  orbit  is 
an  (uncountable)  connected  set.  Thus,  there  exist  homeomorphisms  between  discrete  and 
continuous  orbits  only  when  both  are  constant. 

If  X  is  connected  and  Y  is  a  discrete  topological  space,  this  situation  exists  even  with 
points,  i.e.,  the  only  continuous  functions  from  X  to  Y  are  constant  functions  [113].  One  way 
to  remedy  this  is  simply  to  place  topologies  on  X  and  Y  other  than  their  usual  topologies,  so 
that  continuous  maps  are  possible  (cf.  §3.6).  There  are  several  ways  to  accomplish  this.  One 
approach  is  to  use  so-called  small  topologies  on  X.  Another  is  to  append  a  single  element 
{±}  to  Y,  which  stands  for  “don’t  care”  or  “continue,”  and  topologize  Y'  =  Y  U  {J.}.  For 
more  information  and  other  approaches  see  [24,  114,  §6]. 

Here — and  with  a  view  towards  simulating  systems  defined  on  discrete  topological 
spaces — we  strengthen  the  definition  of  P-simulation  in  two  ways.  First,  we  require  that 
the  “simulated  state”  be  valid  on  some  neighborhood  and  for  at  least  some  minimal  time 
period.  Physically,  this  allows  one  to  use  “imprecise  sampling”  to  obtain  discrete  data, 
providing  a  robustness  that  is  lacking  in  the  definition  of  P-simulation.  Second,  we  require 
that  the  “readout  times”  are  exactly  those  for  which  x(t)  €  i/’~^(Y), 
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Definition  7.3  (I-simulation)  A  continuous  transition  system  [X,E+,/]  simulates  via 
intervals  or  I-simulates  a  discrete  transition  system  [y,Z+,  i^]  if  there  exist  a  continuous 
surjective  partial  function  ip  :  X  Y  and  e  >  0  such  that  V  =  ip  ^  (y )  is  open  and  for^  all 
X  eV  the  set  T  =  {t  \  f{x,t)  G  V"}  is  a  union  of  intervals  {Tk,r'^),  0  =  tq  <  Tq  < 

n  <  <  •  •  •,  Irj^  -  Tfcl  >  e,  with 

ip{f{x,tk))  =  F{ip{x),k), 

for  alltk  e  {Tk^r'if). 

Clearly  I-simulation  implies  P-simulation.  S-simulation  and  I-simulation,  however,  are  in¬ 
dependent  notions. 

The  extra  requirement  that  ip-'^{Y)  be  open  implies  that  the  inverse  images  of  open 
sets  in  Y  are  open  in  X  (and  not  just  in  ip'HY)  as  before).  This  is  probably  too  strong 
a  requirement  in  the  case  of  a  general  topological  space  Y .  However,  in  the  case  of  F  a 
discrete  topological  space,  it  has  the  desirable  effect  that  ip  ^  (y)  is  open  for  all  y  EY. 

One  might  also  have  required  an  output  map  that  is  zero  (or  any  distinguished  output 
value)  on  the  complement  of  T  and  non-zero  otherwise.  This  amounts  to,  in  the  case  of  a 
universal  Turing  machine  simulating  a  machine  M,  the  existence  of  a  distinguished  state 
meaning  “a  step  of  the  simulated  machine  is  not  yet  completed.”  Here,  it  is  related  to 
the  appending  of  a  symbol  {.L}  to  Y  as  above  and  extending  ip  :  X  -^Y'  =  YU  {X}  by 
defining  ip{x)  =  {X}  if  x  €  X\ip-HY)  [7,  24,  114].  In  this  case,  the  requirements  on  iP 
■may  be  replaced  by  requiring  ip  to  be  continuous  from  X  to  Y'  (in  a  suitable  topology) 
after  extension.  Finally,  if  X  is  a  metric  space  one  could  introduce  a  “robust”  version  of 
I-simulation  by  requiring  the  inverse  image  of  y  €  Y  to  contain  a  ball  with  at  least  some 
minimum  diameter. 

Below,  “simulation”  is  a  generic  term,  meaning  I-simulation,  S-simulation,  or  both. 
Si-simulation  denotes  S-simulation  and  I-simulation.  If  a  machine  is  equivalent,  or  simu¬ 
lates  one  that  is  equivalent,  to  a  universal  Turing  machine,  one  says  it  has  the  power  of 
universal  computation. 


§7.3  SIMULATION  WITH  HYBRID  8i  CONTINUOUS  SYSTEMS 

In  this  section  we  concentrate  on  general  simulation  results  and  the  capabilities  of  hybrid 
systems  and  continuous  ODEs. 

We  first  construct  low-dimensional  discrete  dynamical  systems  in  Z”  that  are  equivalent 
to  finite  automata  (FA),  pushdown  automata  (PDA),  and  Turing  machines  (TMs).  Later, 
we  give  some  general  results  for  continuous  ODEs  in  R^n+i  simulating  discrete  dynamical 
systems  in  Z".  Combining  allows  us  to  conclude  simulation  of  arbitrary  FA,  PDA,  and  TMs. 
By  simulating  a  universal  TM,  one  obtains  continuous  ODEs  with  the  power  of  universal 
computation.  In  the  process,  we  also  discuss  the  simulation  and  computational  capabilities 
of  hybrid  systems. 

§7.3.1  DISCRETE  DYNAMICAL  SYSTEMS  EQUIVALENT  TO  FA  AND  TMs 

We  start  by  showing  that  every  TM  is  equivalent  to  a  discrete  dynamical  system  in  Z^  and 
then  consider  systems  equivalent  to  PDA  and  FA.  Later,  we  refine  these  results  to  discrete 
dynamical  systems  in  Z  equivalent  to  TMs,  PDA,  and  FA. 
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The  FA,  PDA,  and  TMs  considered  here  are  deterministic.  Thus  their  transition  func¬ 
tions  naturally  give  rise  to  discrete  dynamical  systems.  These  are  defined  on  state  spaces  of 
input  strings  and  states;  input  strings,  states,  and  stacks;  and  states,  tape  head  positions, 
and  tapes,  respectively. 

Here,  the  states,  input  strings,  stacks,  and  tape  configurations  of  automata  and  Tur¬ 
ing  machines  are  taken  in  the  discrete  topology;  Z"  as  a  topological  or  normed  space  is 
considered  as  a  subspace  of  M"  (in  particular,  it  has  the  discrete  topology). 

See  [76,  §2.1.4]  for  precise  definitions  of  FA,  PDA,  and  TM. 

Proposition  7.4  1.  Every  TM  is  equivalent  to  a  discrete  dynamical  system  in  1? . 

2.  There  is  a  discrete  dynamical  system  in  T?  with  the  power  of  universal  computation. 

3.  Every  FA  and  inputless  PDA  is  equivalent  to  a  discrete  dynamical  system  in  Z.  Every 
PDA  is  equivalent  to  a  discrete  dynamical  system  in  1?. 

Proof. 

1.  Assume  the  tape  alphabet  is  F  =  {7o,7i5  •  •  •  )7m-2}>  m>  2,  with  70  the  blank  symbol; 
and  that  the  set  of  states  is  Q  =  {qo, ,  qn-i}f  n  >  1.  Define  p  =  max{m,n}. 

As  is  customary,  the  one-sided  infinite  tape  is  stored  in  two  stacks,  with  the  state 
stored  on  the  top  of  the  right  stack.  The  coding  used  is  p-ary.  In  particular,  suppose 
the  TM  is  in  configuration  C,  with  tape 

7”  =  7ti )  •  •  •  >  7tAr-i  ’  T’tTv )  5  •  •  • » 

head  positioned  at  cell  N,  and  internal  state  qj.  Encode  the  configuration  C  in  the 
integers 

7V_1  00 

Tl  =  fi{C)  =  P'^^N-k  +  P^{m  -  1),  Tr  =  h{C)  =i  +  Y,P  iN+k- 

fc=0 

The  second  sum  is  finite  since  only  finitely  many  tape  cells  are  non-blank.  The  integer 
(m  -  1)  is  an  end-of-tape  marker.  The  TM  is  assumed  to  halt  on  moving  off  the  left 
of  the  tape,  so  that  (m  -  1,Th)  in  Z^  is  a  fixed  point  for  all  valid  Tr.  On  all  other 
valid  configurations,  C,  define  transition  function  G  in  Z^  by 

G(/i(C),/2(C))  =  (/i(C0,/2(O')), 

where  C  is  the  configuration  resulting  when  the  next  move  function  of  the  TM  is 
applied  to  configuration  C. 

2.  Use  part  1  with  any  universal  TM. 

3.  The  inputless  cases  are  immediate  from  part  1.  For  the  cases  with  input,  note  that 
we  encode  the  input  string  in  an  integer  like  the  left  part  of  the  tape  of  a  TM  above, 
the  results  following. 

□ 

Note  that  one  can  perform  the  above  encodings  of  TMs,  FA,  and  PDA  with  [0,p]  replac¬ 
ing  Z.  Merely  replace  p  by  p~^  in  the  formulas.  The  important  thing  added  is  compactness. 
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and  other  encodings,  e.g.,  with  [0, 1]  replacing  Z,  follow  similarly.  There  is  a  pr^lem  using 
these  encodings  since  two  distinct  tapes  may  have  the  same  encoding,  e.g.,  3, 2, 0“^  and  3, 1". 
One  can  get  around  this  by  “separating”  each  tape  encoding  by  replacing  p  with  2p  and 
using  2i  for  the  zth  symbol.  Namely,  the  tape  of  length  iV,  T  =  7tn  •  •  •  i  Hn  ■>  encoded 
as  Such  Cantor  encodings  were  used  in  [131].  We  still  do  not  use  such 

encodings  here,  however,  since  later  we  want  to  ensure  a  minimum  distance  between  any 
two  tape  encodings. 

Finally,  a  wholly  different  approach  is  to  use  encodings  inspired  by  those  in  [49].  Suppose 
we  are  given  an  arbitrary  TM,  T.  Let  g,  /i,  Z,  and  r  be  integer  codings  of  its  state,  position  of 
its  read-write  head,  the  parts  of  the  tape  on  the  left  and  on  the  right  of  its  heaxl,  respectively. 
A  configuration  of  T  is  encoded  in  the  integer  2®  3^  5^  7”. 

More  generally,  any  discrete  dynamical  system  in  Z”  is  equivalent  to  one  in  Z  by  using 
such  encodings,  viz.,  by  associating  (*1,^2) with  p^iP^i  •  •  -  Pn  ^  where  pi  is  the  ith 
prime. 

We  could  have  used  such  constructions  instead  of  those  in  Proposition  7.4.  However, 
we  retain  them  since  their  transition  functions  have  properties  which  those  arising  from  the 
“prime  encodings”  do  not  (cf.  §7.3.3).  In  any  case,  we  conclude 

Proposition  7.5  Every  TM,  PDA,  inputless  PDA,  FA,  and  inputless  FA  is  equivalent  to 
a  discrete  dynamical  system  in  Z.  There  is  a  discrete  dynamical  system  in  Z  with  the  power 
of  universal  computation. 

It  is  important  to  note  that  one  can  extend  the  transition  functions  in  Z"  above  to 
functions  taking  R"  to  R".  We  may  extend  any  function  /  ;  A  C  Z"  -)■  R"*  in  such  a 
manner,  by  first  extending  arbitrarily  to  domain  Z”  and  then  using  linear  interpolation. 
Here  is  an  example,  used  below: 

Example  7.6  A  continuous  mod  function  may  be  defined  as  follows: 

_  f  ([xj  mod  m)  +  X  —  [x\,  0  <  [xj  mod  m  <  m  —  1, 

X  mode  m  =  I  ^  m  =  m  -  1. 

Later  results  require  extensions  that  are  robust  to  small  input  errors.  That  is,  one  would 
like  to  obtain  the  integer-valued  result  on  a  neighborhood  of  each  integer  in  the  domain. 
For  instance,  one  may  define  a  continuous  nearest  integer  function,  [•]c  :  R  K,  that  is 
robust  in  this  manner  as  follows: 

_  f  z,  i  —  1/3  <  X  <  z  -I- 1/3, 

F]c  =  I  3x_2i_i,  z-i-1/3  <x  <z-l-2/3. 

More  generally,  define  11 :  R”  R" ,  by 

n(x)  =  [[xilc,  •  •  •  >  [a;n]c]- 

Then  given  any  function  /  :  R”  R”*,  with  /(Z")  C  Z"*,  we  can  define  a  “robust  version” 
by  using  the  function  /  o  11. 

Thus,  given  [A,  Z+,  F],  A  C  Z”,  its  transition  function  may  be  extended  to  a  continuous 
function  'from  R"  to  R"  which  is  constant  in  a  neighborhood  of  each  point  in  A.  Such  a 
remark  is  actually  a  byproduct  of  a  more  general  result  needed  below  [113,  p.  216]: 
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Fact  7.7  Any  continuous  function  f  :  A  R'”,  A  a  closed  subset  o/R",  may  be  extended 
to  a  continuous  map  /  ;  R”  — >  R"* . 

Throughout  the  rest  of  this  section  we  use  continuous  extensions  as  in  the  fact  above, 
the  notation  /  always  denoting  such  an  extension  of  / . 

§7.3.2  THE  POWER  OF  EXACT  CLOCKS 

Later,  we  find  that  the  ability  to  implement  precise  timing  pulses  is  a  strong  system  char¬ 
acteristic,  enabling  one  to  implement  equations  with  powerful  simulation  capabilities.  To 
this  end,  define 

Definition  7.8  (Exact  Clock)  A  function  S  is  an  exact  m-ary  clock  with 

pulse-width  T  or  simply  (m,T)-clock  if 

1.  It  is  piecewise  continuous  with  finite  image  Q  =  {0, . . . ,  m  —  1},  m  >  2. 

2.  For  all  t  €  {kT,  {k  +  1)T),  S{t)  =i  if  k  =  i  (mod  m). 

All  reviewed  hybrid  systems  models,  WHS,  TDA,  BGM,  NKSD,  ASL,  and  BAUT,  can 
implement  (m,  r)-clocks,  as  the  results  of  §3.10  and  the  following  shows. 

Example  7.9  1.  The  BAUT  model  implements  {m,T)-clocks:  Choose  Z  =  {0, . . .  ,m  — 

1}  and 

P  =  1/T,  p(0)  =  0, 

z\p]  =  (z[pj -f  1)  modm,  ^[0]  =  0. 

Then  S{t)  —  z[p{t)\  is  an  {m,T)-clock. 

2.  The  TDA  model  implements  {m,T)-clocks:  Setp  =  m  if  m  even,  p  =  m  +  l  ifm  odd. 
Choose  state  space  R  x  Q,  Q  =  {0, . . .  ,p  -  1}.  Define  the  continuous  dynamics  as 

fix,q)  -  c,(-l)’, 

q  EQ.  Set  Cq  =  l  for  all  q  if  m  is  even;  set  Cq  =  I  for  q  e  {0, . . .  ,m  -  2},  c,  =  2  for 
q  ^  —  l,m},  ifm  odd.  In  each  case  define  the  switching  manifolds  by 

92k,2k+l{3:)  =  x  —  T, 

92k+l,{2k+2)mod  p(®)  = 

Setting  x(0)  =  0,  S{t)  =  q{t)  and 

5(t)  =  g(t)-(m-l)[g(<)/3(m-l)]c 

are  {m,T) -clocks  when  m  is  even  and  odd,  respectively. 

As  an  example  of  the  simulation  power  one  obtains  with  access  to  an  exact  clock,  consider 
the  following: 

Theorem  7.10  Every  reversible  discrete  dynamical  system  F  defined  on  a  closed  subset  of 
R”  can  be  S-simulated  by  a  system  of  continuous  ODEs  in  R^”  with  a  {2,T)-clock,  S,  as 
input. 
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Proof. 

x{t)  =  T-'^[G{z)  -  z]{l  -  S{t)), 
at)  =  T-'^[x  -  H{x)]S{t), 

where  G  and  H  are  continuous  extensions  of  G  =  and  H  =  F(',  — 1),  respectively. 

Starting  this  system  at  t  =  0  with  x(0)  =  2;(0)  =  xq,  xq  e  domain  G,  one  sees  that 
x{2kT)  =  z{2kT)  =  G*(xo).  Here,  ipix, 2)  =  x  for  i  =  2,  x  €  domain  G.  □ 

This  theorem  shows  that  exact  clocks  allow  one  to  S-simulate  arbitrary  reversible  discrete 
dynamical  systems  on  closed  subsets  of  R"  with  a  system  of  ODEs  in  R  .  The  idea  of 
turning  on  and  off  separate  systems  of  differential  equations  is  key  to  the  simulation.  The 
effect  of  the  simulation  is  that  on  alternating  segments  of  time  one  “computes”  the  next 
state,  then  copies  it,  respectively.  Then,  the  process  is  repeated.  One  readily  sees  that  the 
exact  way  the  continuous  extensions  in  the  proof  are  performed  is  not  important. 

As  seen  above  each  of  the  reviewed  hybrid  systems  models  can  implement  (2,  T')-clocks. 
In  particular,  they  can  implement  a  (2,T)-clock  with  just  a  single  ODE.  Thus  the  simula¬ 
tions  of  the  theorem  can  be  performed  with  continuous  state  space  in  each  of  these 

cases.  Further,  they  each  require  only  2  discrete  states. 

The  generality  of  Theorem  7.10  allows  us  to  conclude 

Corollary  7.11  Using  S-simulation,  any  hybrid  systems  model  that  implements  continuous 
ODEs  and  a  {2,T)-clock  has  the  power  of  universal  computation. 

Proof.  Using  constructions  as  in  Proposition  7.4,  construct  a  reversible  discrete  dynamical 
system  in  Z"  equivalent  to  a  universal,  reversible  TM  (one  whose  transition  function  is 
invertible)  [13,  138].  In  turn,  simulate  it  using  the  theorem.  ^  □ 

However,  we  want  to  explore  simulation  of  non-reversible  finite  and  infinite  computa¬ 
tional  machines  with  hybrid  and  continuous  dynamical  systems.  First,  we  show  that  the 
ability  to  set  parameters  on  clock  edges  is  strong. 

Theorem  7.12  Every  discrete  dynamical  system  F  defined  on  a  closed  subset  of  R"  can 
be  S-simulated  by  a  system  of  continuous  ODEs  on  R^"  (resp.  R”)  with  a  {2,  T) -clock,  S, 
as  input  and  the  ability  to  set  parameters  on  clock  edges. 

Proof.  Define  G  =  F(-,l).  Both  systems  are  initialized  at  <  =  0  with  c  =  x(0)  =  xo, 
xo  €  domain  G. 

1.  Initialize  2(0)  =  xq.  Use 

x{t)  =  r-nG(z)  -  z](l  -  5(t)), 
z{t)  =  T-Hx-c]5(t). 

The  constant  c  is  set  to  2  when  t  =  kT,  k  odd.  One  sees  that  x{2kT)  =  z{2kT)  — 
G*'(xo).  Choose  tl}{x,  2)  =  x  for  x  =  2,  x  €  domain  G. 

2.  Use 

x{t)  =  T-'[G(c)  -  c](l  -  S{t)). 

The  constant  c  is  set  to  x  when  t  =  kT,  k  even.  One  sees  that  x{2kT) 

Choose  if{x)  =  X,  X  G  domain  G. 


=  G'^ixo). 
□ 
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Note  that  if  F  is  not  reversible,  forward  trajectories  of  the  above  systems  of  equations 
may  merge.  This  situation  is  allowed  by  our  definitions.  The  simplest  example  of  this  is 
[{0, 1},  Z+,  F]  with  F(0, 1)  =  F(l,  1)  =  0. 

Corollary  7.13  Any  hybrid  systems  model  that  implements  continuous  DDEs,  a  (2,T)- 
clockf  and  setting  parameters  on  clock  edges,  can  S-simulate  any  TM,  PDA,  or  FA;  and, 
using  S-simulation,  has  the  power  of  universal  computation. 

Proof.  Combine  the  theorem  and  Proposition  7.4.  □ 

In  particular,  the  BGM  model  has  this  power  (by  defining  the  appropriate  transition 
functions  on  the  switching  boundaries  of  the  TDA  (2,T)-clock  given  above). 

§7.3.3  SIMULATION  WITHOUT  EXACT  CLOCKS 

Without  an  exact  clock,  one’s  simulation  power  is  limited.  However,  one  can  still  simulate 
discrete  dynamical  systems  defined  on  arbitrary  subsets  of  Z”.  Next,  we  proceed  to  explicitly 
show  that  all  hybrid  systems  models  of  §3  can  simulate  any  discrete  dynamical  system  on 
Z".  Indeed,  we  show  that  continuous  ODEs  can  simulate  them. 

In  the  previous  section  we  used  an  exact  (2,  r)-clock  to  precisely  switch  between  two 
different  vector  fields  in  order  to  simulate  discrete  dynamical  systems  in  R".  Again,  the 
essential  idea  behind  the  simulations  in  this  section  is  to  alternately  switch  between  two 
different  vector  fields.  However,  since  we  are  simulating  systems  in  Z",  using  “robust  ver¬ 
sions”  of  their  transition  functions,  and  choosing  well-behaved  ODEs,  it  is  not  necessary  to 
precisely  time  these  switches  using  an  exact  clock.  Indeed,  we  can  use  continuous  functions 
to  switch  among  vector  fields. 

It  is  still  convenient  to  ensure,  however,  that  only  one  vector  field  is  active  (non-zero) 
at  any  given  time.  Thus,  we  would  like 

Definition  7.14  (Inexact  Clock)  An  inexact  (m,T)-clock,  m  >  2,  is  a  continuous 
function  5  :  R+  [0, 1]"*  such  that  on  each  interval  t  €  [kT,  {k  +  l)r]  with  k  =  i  (mod  m) 
the  following  hold:  Sj+iit)  =  0,  0  <  j  <  m  -  1,  j  i;  5i+i(t)  =  1  on  a  sub-interval  of 
length  greater  than  or  equal  to  r/2. 

It  is  also  reasonable  to  require  that  transitions  between  0  and  1  take  place  quickly  or 
that  there  be  some  minimum  separation  between  the  times  when  Si  >  0,  Sj  >  0,  i  ^  j. 
Below,  we  need  an  inexact  (2,r)-clock  with  the  latter  property. 

What  is  key  is  that  such  inexact  clocks  do  not  require  discontinuous  vector  fields,  dis¬ 
continuous  functions,  or  discrete  dynzunics.  They  can  be  implemented  as  follows. 

Example  7.15  (Inexact  (2,T)-clock)  Define  f(<)  =  l/T,  initialized  at  t(0)  =  0.  Now, 
define 

5i,2(t)  =  h±[sm{nT)], 

where 

(  0,  r<S/2, 

/i+(r)  =  '{  2r/J-l,  5f2<r<6, 

[h  S<r, 

h-{r)  =  h+{—r),  and0  <  5  <  \/2l2. 

Thus,  one  can  switch  between  two  different  systems  of  ODEs  with  (Lipschitz)  continuous 
functions  of  the  state  of  another  (Lipschitz)  ODE.  This  is  why  2n  -I- 1  dimensional  ODEs 
are  used  below  to  simulate  an  n-dimensional  discrete  dynamical  system. 
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We  also  need  the  following  technical  definitions: 

Definition  7.16  (Non-degeneracy,  Finite  Gain)  A  function  f  :  W  -¥  W ,  is  non- 
degenerate  (resp.  finite  gain)  if  there  exist  constants  /3  >  0,  M  >  0,  such  that 

Ikll  <  M\\f{x)\\  +  A  (resp.  l|/(x)||  <  M||a:l|  +  /3), 

for  all  X  E  X. 

Now  we  are  ready  for  our  main  simulation  result: 

Theorem  7.17  Every  discrete  dynamical  system  F  defined  onY  C 

1.  can  he  Sl-simulated  by  a  system  of  continuous  ODEs  in  . 

2.  such  that  F(-,  1)  is  finite  gain  and  non-degenerate  can  be  I-simulated  by  a  system  of 
continuous  ODEs  in  . 

3.  such  that  Y  is  bounded  can  be  Sl-simulated  by  a  system  of  Lipschitz  ODEs  in  . 

Proof.  Let  G  =  F{-,  1)  and  0  <  e  <  1/3.  5i,2  and  6  are  as  in  the  preceding  example.  For 
each  y  €Y,  define  the  set 

Hy  =  {(x,  2,  t)  I  llx  -  ylloo  <  e,  11^  -  y||oo  <  e,  sin(7rT)  <  S/2,  t  modc2  <  1/2}, 

Set  ^(x,  2,  r)  =  n(2)  =  y  if  (x,  2,  r)  €  Hy.  Note:  the  =  Hy  are  open  and  disjoint. 

Initialize  x(0),  2(0),  t(0)  in  y  €Y. 

1.  Choose 

1  =  _e-2[a:-G(n(2))]3Sx(T), 

2  =  -e-2[2-n(x)f52(r), 

f  =  1. 

It  is  straightforward  to  verify  n(2(2A:))  =  G''(y),  k  £  Z+,  and  the  interval  constraint. 

2.  Let  a  and  L  be  the  finite  gain,  and  /?  and  M  the  non-degeneracy  constants  of  G  under 
norm  |1  •  ||oo-  Choose 

X  =  -2€-Mx-G(n(2))]5i(r), 

i  =  -  n(x)]52(T),  (7.1) 

f  =  1/[H- (L -1- l)ll2:l|c» +  «+ + l)ll®lloo  + /?]  • 

It  is  straightforward  to  verify  n(2(t))  =  G^{y)  on  an  interval  about  the  time  tk  where 
T(tfc)  =  2fc,  k  €  Z4.. 

3.  Let  0  =  max{lli  -  j\\oo  1  hj  €  F}.  Choose 

X  =  -2/3e-Mx-G(n(2))]5i(T), 

2  =  -2/?e-i[2-n(x)]52(T),  (7.2) 

T  =  1. 

It  is  straightforward  to  verify  U{z{2k))  =  G''(y),  fc  €  Z+,  and  the  interval  constraint. 
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Note  that  non-degeneracy  and  finite  gain  of  the  extension  G  need  not  hold  for  points 
not  in  Y.  Note  also  that  the  simulations  above  are  “robust”  in  the  sense  that  there  is  a 
neighborhood  of  initial  conditions  leading  to  the  correct  simulations.  The  import  of  part  2 
of  the  theorem  is  that  if  G  =  F(-,  1)  is  non-degenerate  and  may  be  extended  to  a  Lipschitz 
function,  then  the  ODEs  used  in  the  I-simulation  are  also  Lipschitz. 

Note  also  that  the  theorem  continues  to  hold  for  any  discrete  dynamical  system  defined 
on  y  C  K"  such  that  there  is  some  minimum  separation  between  any  two  distinct  points  of 

y. 

The  discrete  dynamical  systems  equivalent  to  TMs  given  by  Proposition  7.4  have  tran¬ 
sition  functions  that  are  both  finite  gain  and  non-degenerate.  Unfortunately,  the  transition 
functions  of  systems  equivalent  even  to  PDA  need  not  be  Lipschitz.  Consider  a  PDA  which 
pushes  a  tape  symbol  7  on  input  symbol  h  and  pops  7  on  input  symbol  ^2  and  test  with 
inputs  of  the  form  and  One  may  check  that  the  “prime  encodings”  mentioned 
earlier  lead  to  transition  functions  that  are  neither  finite  gain  nor  non-degenerate. 

Nevertheless,  relating  the  theorem  back  to  simulation  of  TMs,  PDA,  and  FA,  we  have 
many  results,  the  most  striking  of  which  are: 

Corollary  7.18  Every  TM,  PDA,  and  FA  can  be  Sl-simulated  by  a  system  of  continuous 
ODEs  in  . 

Every  FA  (resp.  inputless  FA)  can  be  I-simulated  (resp.  Sl-simulated)  by  a  system  of 
Lipschitz  continuous  ODEs  in  . 

Using  Sl-simulation,  there  is  a  system  of  continuous  ODEs  in  with  the  power  of 
universal  computation. 

Proof.  Everything  is  immediate  from  the  theorem  and  Propositions  7.4  and  7.5  except 
that  the  FA  transition  function  is  Lipschitz,  which  is  readily  checked.  □ 

Of  course,  any  hybrid  systems  model  that  implements  continuous  (resp.  Lipschitz)  ODEs 
has  similar  powers.  In  particular,  the  models  reviewed  in  §3  do. 

Finally,  all  the  simulation  results  for  discrete  dynamical  systems  on  Z  can  be  extended 
from  continuous  to  smooth  vector  fields  by  using  C°°  interpolation  (with  so-called  bump 
functions  [59])  rather  than  linear  interpolation  in  extending  their  transition  functions  and 
the  functions  [■]c  and  h±,  and  by  replacing  ||  •  ||oo  with  \\  •  II2  in  Equation  (7.1). 


§7.4  IMPLEMENTING  ARBITERS 

In  this  section,  we  contrast  the  capabilities  of  hybrid  and  continuous  dynamical  systems  by 
using  the  famous  asynchronous  arbiter  problem  [26,  100,  144]. 

We  begin  in  the  first  subsection  with  a  discussion  of  the  arbiter  problem.  Next,  we  prove 
that  one  cannot  implement  an  asynchronous  arbiter  using  a  system  of  Lipschitz  ODEs 
continuous  in  inputs  and  outputs,  i.e.,  a  system  of  the  form  of  Equation  (2.2)  with  / 
Lipschitz  in  x,  continuous  in  u  and  h  continuous  [73,  p.  297].  Finally,  we  show  that  all 
hybrid  systems  models  in  §3  can  implement  arbiters,  even  when  their  continuous  dynamics 
is  a  system  of  Lipschitz  ODEs  continuous  in  inputs  and  outputs. 

§7.4.1  THE  ARBITER  PROBLEM 

The  definition  and  technical  specifications  of  an  (asynchronous)  arbiter  below  are  adapted 
from  [144]. 
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An  arbiter  is  a  device  that  can  be  used  to  decide  the  winner  of  two-person  races.  It  is 
housed  in  a  box  with  two  input  buttons,  labeled  Bi  and  B2,  and  two  output  lines,  Wi  and 
W2,  that  can  each  be  either  0  or  1.  For  ease  of  exposition,  let  the  vectors 

B  =  (5i,B2),  W  =  {Wi,W2) 

denote  the  button  states  and  outputs,  respectively.  There  is  also  a  reset  button,  R.  Below, 
the  buttons  Bi,  R  axe  taken  to  be  1  when  they  are  pressed,  0  when  they  are  unpressed. 

After  the  system  has  been  reset,  the  output  should  be  (1,0)  if  Bi  is  pressed  before  B2', 
it  should  be  (0, 1)  if  B2  is  pressed  before  Bi.  Let  Ti  denote  the  time  that  button  Bi  is 
pressed.  Then,  the  function  of  the  arbiter  is  to  make  a  binary  choice  based  on  the  value  of 
the  continuous  variable  Ti  -  T2.  If  the  difference  is  negative,  the  output  should  be  (1,0);  if 
it  is  positive,  the  output  should  be  (0, 1).  Upon  reset,  the  output  is  set  to  (0,0). 

Here  are  the  arbiter’s  technical  specifications: 

51.  Pressing  the  reset  button,  R,  causes  the  output  to  become  (0, 0),  perhaps  after  waiting 
for  some  specified  time,  denoted  Tr,  where  it  remains  until  one  or  both  buttons  are 

pressed. 

52.  The  pressing  of  either  or  both  buttons  Bi  causes,  after  an  interval  of  at  most  Td  units, 
the  output  to  be  either  (0,1)  or  (1,0);  the  output  level  persists  until  the  next  reset 

input. 

53.  If  Bi  is  pressed  Ta  seconds  or  more  before  B2  is  pressed,  then  the  output  is  (1,0), 
indicating  that  Bi  was  pressed  first.  Similarly,  if  B2  is  pressed  Ta  seconds  or  more 
before  Bi  is  pressed,  then  the  output  is  (0, 1),  indicating  that  B2  was  pressed  first. 

54.  If  Bi  and  B2  are  pressed  within  Ta  seconds  of  each  other,  then  the  output  is  either 
(1,0)  or  (0, 1) — one  does  not  care  which — after  the  Tj-second  interval. 

The  arbiter  problem  is 

Problem  7.19  (Asynchronous  arbiter  problem)  Build  a  device  that  meets  the  specifi¬ 
cations  S1-S4. 

§7.4.2  YOU  CAN’T  IMPLEMENT  AN  ARBITER  WITH  LIPSCHITZ  ODEs 

In  this  section,  we  show  that  it  is  impossible  to  build  a  device,  described  as  a  systena  of 
Lipschitz  ODEs  continuous  in  the  required  inputs  and  outputs,  that  implements  the  arbiter 
specifications. 

First  we  give  a  generic  system  of  Lipschitz  ODEs  with  the  required  properties: 

x{t)  =  fix{t),B{t)), 

W{t)  =  h{xit)), 

where  x{t)  e  1”,  W{t)  G  R^,  B{t)  E  {O,!}^,  with  J5(-)  piecewise  continuous.  Each  /(^B), 
B  e  {0,1}2,  is  Lipschitz.  Thus,  each  vector  field  /(^B)  defines  a  continuous  dynamical 
system  b^),  with  the  solution  at  time  T  of  i:(t)  =  f  {x{t),  Bi,  B2)  starting 

at  x(0)  =  x  'q.  Further,  /i :  R”  — >  R^  is  continuous.  Note  that  the  action  of  the  reset  button 
is  unmodeled;  it  is  not  necessary  to  the  proof,  which  assumes  it  remains  unpressed  on  the 
interval  of  interest. 
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Since  h  is  continuous,  there  exists  a  constant  >  0  such  that 

\\h{x)  -  h{x')\\2  <  \/2  whenever  ||x  -  x'H  <  5^,  (7.4) 


Define  Lw  = 

Now,  we  are  reaxly  to  settle  the  arbiter  problem  in  this  framework: 

Theorem  7.20  For  no  choice  of  the  values  for  Ta  and  Td  is  it  possible  to  build  a  device 
described  by  Equation  (7.3)  that  meets  the  arbiter  specifications  S1-S4. 


Proof.  The  proof  is  by  contradiction,  assuming  there  is  a  device  described  by  Equation 
(7.3)  which  satisfies  the  specifications. 

Assume  that  the  arbiter  has  been  reset,  is  in  state  x(0)  =  xq  at  time  t  =  0  with 
h{xo)  =  (0,0),  and  that  one  of  the  buttons  is  pressed  at  time  t  =  0.  (This  is  without  loss 
of  generality  as  the  equations  axe  autonomous.)  Also,  assume  that  the  reset  button  is  not 
pressed  until  some  time  Tr^  Ta  +  Td- 

The  behavior  of  the  device  from  f  =  0  to  t  =  Tr  is  completely  determined  by  which 
button  was  pressed  first  and  at  what  time  the  second  button  is  pressed  (if  ever).  Therefore, 
let  Xp(t)  denote  the  solution  at  time  t  of  Equation  (7.3)  starting  at  time  t  =  0  at  state 
x(0)  =  xo  with  fixed  parameter  p  =  Ti  -  T2.  Thus,  p  represents  the  difference  between  the 
times  when  JBi  and  B2  are  pressed.  If  Bi  is  pressed  but  B2  is  never  pressed,  set  p  =  -00. 
If  B2  is  pressed  but  Bi  is  never  pressed,  set  p  =  00. 

The  arbiter  specifications  require  that  for  To  +  Td  <  t  <  Tr, 


h(xp(t))  =  I 


(1,0), 

(0,1), 

(1,0)  or  (0,1), 


P  ^  To, 

P>Ta, 

otherwise. 


These  specifications  and  Lemma  7.23  (in  the  Appendbc)  are  such  that  for  any  5  >  0,  one 
can  find  -To  <  ct  <  t  <  To,  with  t-  ct  <  5,  and  with  one  of  h(x^{Ta  +  Td)),  h(xr(Ta  +Td)) 
equal  to  (1,0)  and  the  other  equal  to  (0, 1). 

Pick  S  <  min{Ta,Td,  1/L},  where  L  >  0  is  a  finite  bound  of  the  maximum  of  the  four 
Lipschitz  constants  corresponding  to  each  of  the  f{-,B).  Define 

c  =  max{||/(xo,  1,0)|1,  |l/(xo,0, 1)||,  ||/(xo,  1, 1)1|}  . 


Note  c  >  0,  for  otherwise  h(x<^(t))  =  h(xr{t))  =  h{xo)  for  all  0  <  f  <  Tr,  a  contradiction. 

For  ease  of  notation,  let  Ft,  Gt,  Ht  denote  the  fundamental  solutions  (f>(o,i)(-,t),  </>(i,o)  (•,*), 
and  <f>[i^i){-,t),  respectively.  Also,  let  Xt  =  Xtr{t)  and  Yt  =  Xr(t).  Note  that  Aq  =  1^  =  xq. 
The  proof  splits  into  three  cases: 

1.  0  <  <T  <  r  <  Ta. 

2.  -Ta  <  <r  <  T  <  0. 

3.  -Ta  <cr  <0<T  <Ta. 


Case  1.  In  this  case. 


Ft(xo),  0<t<a, 

Ht_<,(F,(xo)),  (r<'t<TR, 
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Ft{xo),  0<t<T, 

Ht-riFr{xo)),  T<t<TR. 


Thus,  Xa  =  Yc.  Now,  by  Corollary  7.25 

\\Yr-Ya\\<cL-\e^^ 


Thus,  Lemma  7.26  gives 

||X,+Ta-n+Tjl  < 

<  cS{e  - 

where  the  last  line  follows  from  L5  <\.  But  by  assumption, 

v/2  =  i|/l(X,+Ta)-/l(^T+Tjli2, 

SO  that  Equation  (7.4)  yields 

ATi  =  l[cLw{e  -  <  8. 

Case  2.  The  argument  is  similar  to  Case  1  and  yields  the  same  inequality  on  8. 
Ceise  3.  In  this  case. 


Xt  = 
Ft  = 


Gt(xo),  0  <  t  <  |(7|, 

^rt_|<r|(G|<,|(xo)),  \(T\<t<TR, 

Ftixo),  0  <  t  <  T, 

Ht-TiFrixo)),  T  <  t  <  Tr. 


Note  that  max{lal,  |rl}  <  8.  This  and  Lemma  7.24  give 

Iku,  -  lol  +  \\Y,  -  loll  <  2ci-‘(e"  -  1) 


IFki  -  M  s 

Thus,  Lemma  7.26  gives 

ll-X'i^Tl+rd  -  Fr+Tdl  ^  2cL 

<  2ce^^‘'  (e  —  1)5, 

where  the  last  line  follows  from  L5  <  1.  But  by  assumption, 

(-X^ltrl+Td)  ~  ^  (Fr+Ta) [jj  i 

so  that  Equation  (7.4)  yields 

Kz  =  \  j  ^■y/2cXw(e  —  l)e^^‘'j  <  8. 

Thus,  choosing  8  <  min{Ta,rd,  would  have  achieved  a  contradiction  in  all 
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three  cases.  D 

The  basic  argument  used  above  is  that  one  cannot  have  a  continuous  map  from  a  con¬ 
nected  space  (e.g.,  R  containing  p)  to  a  disconnected  space  (e.g.,  {(1,0),  (0,1)})  [113]. 
Nevertheless,  one  must  prove  that  the  map  given  by  the  device  is  indeed  continuous  before 
one  makes  such  an  appeal.  Above,  we  have  explicitly  demonstrated  the  continuity  of  the 
system  of  switched  differential  equations  describing  the  arbiter. 

§7.4.3  IMPLEMENTING  ARBITERS  WITH  HYBRID  SYSTEMS 

In  this  section  it  is  shown  that  each  of  the  hybrid  systems  models  can  implement  an  arbiter. 
Given  the  results  of  §3.9,  it  is  enough  to  implement  one  using  the  BAUT  and  TDA  models. 
However,  the  problem  is  such  that  we  must  add  inputs  and  outputs  to  these  models,  which 
is  done  in  an  obvious  way. 

In  each  case,  the  continuous  dynamics  is  a  system  of  Lipschitz  ODEs  continuous  in  inputs 
and  outputs,  the  essential  “resolving  power”  coming  from  the  mechanisms  implementing  the 
discrete  dynamics. 

We  first  implement  an  arbiter  with  a  hybrid  system  a  la  Brockett: 

Proposition  7.21  There  exists  a  system  of  equations  in  the  BAUT  model  with  inputs  and 
outputs  that  meets  the  arbiter  specifications  S1-S4. 

Proof.  We  design  for  Ta  =  Td/2  =  Tm- 

X  =  [2{4z\j>\  -  1)  max{BuB2)T{x)/Tm]  {l-R)-  {2x/Tr)  R, 
p  =  [2Bi{Bi-B2){l-z\p\)/Tm]{l-R)  +  {z[p\/TR)R, 

=  (•^bJ  + 1)  2, 

W  =  h(x), 

where 

'(0,1),  x<-3, 

(0,|x|-2),  -3<x<-2, 
h{x)  =  •{  (0,0),  -2<x  <2, 

(x-2,0),  2<x<3, 

.(1,0),  3<x, 

'  1,  b|  <  4, 

T{x)  =  <  5  -  |x|,  4  <  |x|  <  5, 

0,  5<|x|. 

Let’s  examine  these  equations  when  B2  is  pressed  at  time  t  =  0.  Let  Ti  >  0  denote  the  time 
at  which  Bi  is  pressed.  The  equations  are  assumed  to  be  properly  reset  so  that  without 
loss  of  generality,  we  assume  that  |x(0)|  <  1  and  p(0)  €  [2fc,  2fc  +  1),  for  some  k  €  Z+, 
and  2;b(0)J  =  0.  Also,  we  assume  that  the  reset  button  is  inactive  (i?  =  0)  from  t  =  0  to 
t  =  tR>  2Tm-  In  this  case,  the  two  equations  are  simply  (no  matter  when  Bi  is  pressed) 

X  =■  2!T{x)/Tmf 

p  =  0, 

so  that  x{t)  <  -3  and  hence  W{t)  =  (0, 1)  for  t  €  [2Tm,tR]. 

Now,  we  look  at  these  equations  under  the  same  assumptions,  excepting  Bi  is  pressed 
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at  t  =  0  and  B2  is  pressed  at  t  =  T2  >  0.  Now  there  are  two  cases:  T2  <  U  and  T2  > 
where  <,  =  [1  -  (p  -  bJ)]^m/2  <  T^/2  is  the  time  when  z{[p{t)\)  would  first  equal  1  if  B2 
were  not  pressed  before  it.  In  the  second  case,  by  time  tz  the  equations  are 

X  =  6T{x)lTm, 

p  =  0, 


so  that  x{t)  >  4  and  hence  W{t)  =  (1,0)  for  te[tz  +  7m,fR]  [2Tm,fii].  In  the  first  case, 
the  first  equation  remains 

X  =  -2Tix)ITm, 

so  that  x{t)  <  -3  and  hence  W{t)  =  (0, 1)  for  t  G  [2Tm,tR]. 

The  reset  behavior  is  readily  verified.  '-I 

Now,  we  implement  an  arbiter  with  TDA: 


Proposition  7.22  There  exists  a  system  of  equations  in  the  TDA  model  with  inputs  and 
outputs  that  meets  the  arbiter  specifications  S1-S4. 

Proof.  For  convenience,  define  T„,  =  mm{Td,Ta}.  Define  the  continuous  dynamics, 
/(x,  q,  Si,  ^2,  R),  which  depends  on  states  x  €  g  €  {1, 2, 3},  and  inputs  Bi,  B2,  and  R, 
each  in  {0, 1},  as  follows: 


/(x,  1;Bi,B2,0) 
/(x,2;-,-,0) 
/(x,3;-,-,0) 

/ (^1  ‘i  *5  '1  1) 


(Bi[Bi-B2],B2), 

(0,0), 

(0,0), 


with  switching  boundaries  defined  as  follows: 


^I2(x)  =  4e2-llx-(T,„,0)ill, 

Pi, 3(2^)  =  X2-Tm, 

52,1(2:)  =53,1(2;)  =  e^-WxWl, 


where  0<e<Tm/'i-  Finally,  define  the  output  W  =  h{x)  where 

'  (1,0),  X2<Tml2, 

/i(x)  =  -^  (l-4(x2/T,„-l/2),4(x2/T,n-l/2)),  W2  <  X2  <  3TW4, 

(0,1),  ^Tm/4<X2. 

One  readily  verifies  that  it  behaves  correctly. 


□ 


§7.5  DISCUSSION 

We  now  turn  to  some  discussion.  Our  simulation  of  arbitrary  Turing  machines  imply  that, 
in  general,  questions  regarding  the  dynamical  behavior  of  hybrid  systems  with  continuous 
ODEs— and  even  well-behaved  ODEs  themselves— are  computationally  undecidable.  See 
[107,  108]  for  a  discussion  of  such  questions.  Further,  the  ODEs  sirnulating  these  machines 
may  be  taken  smooth  and  do  not  require  the  machines  to  be  reversible  (cf.  [108,  p.  228]). 
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The  import  of  S-simulation  here  is  that  such  simulations  take  only  “linear  time”  [49]. 
The  import  of  I-simulation  is  that  the  readout  times  for  which  the  state/tape  is  valid  are 
non-empty  intervals.  Indeed,  the  intervals  are  at  least  some  minimum  length.  Also,  the 
simulations  were  “robust”  in  the  sense  that  they  can  tolerate  small  errors  in  the  coding  of 
the  initial  conditions.  Though  not  required  by  our  definitions,  these  contained  balls  of  at 
least  some  minimum  diameter. 

The  explicit  formulation  and  solution  of  the  asynchronous  arbiter  problem  in  an  ODE 
framework  appears  to  be  new.  One  should  note  that  in  our  ODE  model,  the  inputs  Bi  were 
assumed  to  be  ideal  in  the  sense  that  they  switch  from  0  to  1  instantaneously.  Imposing 
continuity  assumptions  on  5  as  signals  in  [0, 1]^  leads  to  a  similar  result. 

To  demonstrate  the  computational  capabilities  of  hybrid  and  continuous  dynamical  sys¬ 
tems  summarized  above,  we  constructed  low-dimensional  discrete  dynamical  systems  in  Z” 
equivalent  to  Turing  machines  (TMs),  pushdown  automata  (PDA),  and  finite  automata 
(FA).  It  is  well-known  that  certain  discrete  dynamical  systems  are  equivalent  to  TMs  and 
possess  the  power  of  universal  computation  (see,  e.g.,  [49, 107, 131]).  Our  systems  were  con¬ 
structed  with  the  goal  of  simulation  by  continuous/Lipschitz  ODEs  in  mind.  One  notes  that 
while  it  is  perhaps  a  trivial  observation  that  there  are  systems  of  (Lipschitz)  ODEs  with 
the  power  of  universal  computation— just  write  down  the  ODEs  modeling  your  personal 
computer — this  requires  a  system  of  ODEs  with  a  potentially  infinite  number  of  states. 

The  best  definition  of  “simulation”  is  not  apparent.  While  stated  in  terms  of  our  defini¬ 
tions  of  simulation,  the  simulation  results  of  §7.3  are  intuitive  and  would  probably  continue 
to  hold  under  alternate  definitions  of  simulation. 

Related  to  our  general  simulation  results  is  a  theorem  by  N.  P.  Zhidkov  [157]  (see  also 
[129,  p.  135]),  that  states  if  a  reversible  discrete  dynamical  system  is  defined  on  a  compact 
subset  K  CW,  then  there  exists  on  a  subset  of  a  reversible  continuous  dynamical 

system  that  is  defined  by  ODEs  and  has  if  as  a  global  section. 

It  is  possible  to  take  a  diff'erent  approach  than  the  one  in  §7.3  and  construct  smooth 
systems  of  ODEs  with  inputs  that  “simulate”  finite  automata.  For  instance,  in  [36]  Brockett 
used  a  system  of  his  so-called  double-braxJcet  equations  (also  see  [37])  to  “simulate”  the  step)- 
by-step  behavior  of  a  FA.  This  was  done  by  coding  the  input  symbols  of  the  FA  in  a  fimction 
of  time  that  is  the  “control  input”  to  a  system  of  double-bracket  equations.  Specifically,  if 
the  input  alphabet  is  /  =  {ui, . . . ,  Wm},  the  input  string  «fo,  Ujj ,  Ujj, . . .  is  encoded  in  a  time 
function,  u{t),  that  is  h  on  the  intervals  [2kT,  {2k  +  l)r]  and  zero  otherwise.  In  this  paper, 
we  encoded  the  full  input  string  in  the  initial  condition  of  our  simulations. 

In  [36],  Brockett  was  interested  in  the  capabilities  of  his  double-bracket  equations.  How¬ 
ever,  the  resulting  “simulations”  of  FA  happen  to  behave  poorly  with  respect  to  our  defi¬ 
nitions  of  simulation.  Nevertheless,  the  key  idea  of  his  simulations  of  FA  is  that  the  input 
coding,  u(f),  is  used  in  such  a  way  that  it  alternately  switches  between  two  different  systems 
of  double-bracket  equations.  This  idea  is  critical  in  our  simulations  of  discrete  dynamical 
systems  with  ODEs. 

It  is  not  hard  to  see  that  one  could  use  the  same  approach  as  that  in  [36]  but  more 
well-behaved  systems  of  ODEs  to  simulate  the  step-by-step  behavior  of  FA.  Consider  a  FA 
with  transition  function  6,  states  Q  =  {gi, . .  ■  ,gn}>  and  input  alphabet  I  as  above.  Code 
state  Qi  as  i  and  consider  the  first  two  equations  of  Equation  (7.2).  Choose  13  =  n  and 
replace,  respectively.  Si,  Sa,  and  G  with  h+{u{t)),  h-{u{t)  —  1),  and 

D  :  {!,..., n}  x  m}  {!,...  ,n}. 
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defined  by  D{iJ)  =  fc  if  S{quUj)  =  qk-  The  result  is  that  any  FA  may  be  Sl-simulated  by 
a  system  of  ODEs  in  with  input.  This  was  also  announced  in  [22]. 

In  [7],  it  is  shown  that  so-called  piecewise-constant  derivative  systems  (PCDs)  in 
R^  can  “simulate”  arbitrary  inputless  FA,  inputless  PDA,  and  TMs.  Briefly,  the  notion  of 
simulation  used  is  that  of  I-simulation  excepting  as  follows.  First,  the  intervals  in  T  can  be 
open,  closed,  or  half-closed;  ‘<’  may  replace  ‘<’  in  the  constraints  on  t*,,  t(;  and  there  is  no 
e  constraint.  Also,  there  is  no  continuity  constraint  on  ip  and  for  each  y€Y  there  need  exist 
only  one  point  in  ip~^{y)  for  which  the  equation  holds.  However,  there  is  the  constraint  that 
each  ip~^{y)  is  convex  and  relatively-open  (i.e.,  open  in  the  subspace  of  its  affine  hull).  For 
convenience,  we  refer  to  this  notion  as  AM-simulation.  Since  our  I-simulations  in  Theorem 
7.17  had  ip~^{y)  open  and  convex,  they  are  AM-simulations  (here  we  are  thinking  of  t  in 
R/2Z,  or  in  a  circle  embedded  in  R^,  with  appropriate  changes). 

Convexity  of  ip~'^{y)  may  be  a  desirable  property.  For  instance,  it  excludes  simulation 
of  FA  by  “unraveling”  their  transition  diagrams  into  trees,  a  simple  example  of  which  is 
recounted  in  [7].  On  the  other  hand,  consider  the  case  of  a  universal  TM,  U,  simulating 
an  inputless  FA,  A.  Certainly,  there  could  be  many  distinct  configurations  of  U  in  which 
the  current  state  of  A  is  written  on,  say,  its  first  tape  cell.  Then,  even  if  the  inverse  images 
of  the  configurations  of  U  are  convex,  the  inverse  images  of  the  valid  configurations  with, 
say,  q  in  the  first  tape  cell  need  not  be,  preventing  indirect  AM-simulation  of  A  through 
AM-simulation  of  U.  In  any  case,  we  could  have  added  the  constraint  that  each  ip  ^{y)  be 
convex  to  our  definitions  of  simulation  with  little  change  in  any  of  our  results. 

Finally,  in  [7]  Asarin  and  Maler  use  three-dimensional  PCDs  to  AM-simulate  inputless 
FA.  They  also  point  out  that  three  dimensions  are  necessary  in  order  to  AM-simulate, 
with  autonomous  ODEs,  inputless  FA  whose  transition  graphs  are  not  planar.  While  their 
argument  is  fine,  the  transition  graphs  of  deterministic  inputless  FA  are  always  planar  and 
it  is  straightforward  to  construct  PCDs  (and  continuous  ODEs)  in  two  dimensio^  that 
AM-simulate  such  FA.  Moreover,  even  though  the  transition  graphs  of  FA  (with  inputs) 
need  not  be  planar,  their  argument  does  not  contradict  the  result  in  R^  derived  in  this 
section,  since  it  uses  non-autonomous  ODEs. 


§7.6  NOTES 

The  work  in  this  chapter  appeared  in  [31].  Our  simulation  of  arbitrary  Turing  machines 
was  first  announced  in  [22]. 

Prof.  John  Wyatt  presented  the  arbiter  problem  and  the  challenge  to  produce  an  ODE- 
based  model  and  proof  in  his  nonlinear  systems  course  at  MIT  [153].  The  explicit  formula¬ 
tion  and  solution  of  the  asynchronous  arbiter  problem  in  an  ODE  framework  is  excerpted 
from  [26],  which  also  discusses  bounds  on  the  performance  of  systems  approximating  ar¬ 
biter  behavior,  arising  from  the  explicit  proof.  Specifically,  while  the  proof  prohibits  the 
construction  of  an  arbiter  with  Tj  =  0(1),  it  does  not  prohibit  an  arbitration  device  with 
Td  =  0(ln(l/p)).  Such  a  device  is  given  in  [26]. 

Dr.  Charles  Rockland  has  recently  brought  to  our  attention  an  interesting  example  of 
the  complexity  that  can  arise  in  low-dimensional  differential  equations.  In  particular,  [125] 
gives  a  non-trivial  fourth-order  algebraic  differential  equation  (given  by  a  polynomial  with 
integer  coefficients)  exhibiting  smooth  solutions  dense  in  C(R,R). 
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§7.7  APPENDIX 

Lemma  7.23  If  X  is  a  connected  metric  space,  Y  is  a  discrete  topological  space  with  two 
points,  and  f  :  X  Y  is  surjective,  then  for  every  6  >  0  one  can  find  x,z  E  X  such  that 
d{x,z)  <5  and  f{x)i^  f{z). 

Proof.  Assume  the  contrary.  Then  for  all  x  6  X,  f{Bs{x))  =  {/(x)}  C  V ,  where  Bs(x) 
denotes  the  ball  of  radius  S  about  x  and  V  is  any  open  set  about  /(x)  in  Y .  Thus,  /  is 
continuous  [113].  But  /  continuous  and  X  connected  implies  f{X)  =Y  is  connected  [113], 
a  contradiction.  CD 


Lemma  7.24  Suppose  x(t)  =  f{x{t))  with  f  globally  Lipschitz  continuous  in  x  with  con¬ 
stant  Lf  >0.  Then,  for  any  L  such  that  L>  Lf  and  L>  0,  and  any  t2>ti, 

IK  -  xtji  <  ||/(xt,)|lL-Me^(‘-‘>)  -  1). 


Proof.  Note  that  for  t>ti, 


So  that 


Xt  Xf^  — 


jja^t  -  a:ti  II  < 
< 


f  f{xn)ds  +  f  [f{x^)-  f{xn)]ds. 

Jti  Jti 

I  ll/(a;ti)ll«^s+  /  Wfi^s)- f{xti)\\ds 
Jti  Jti 

(*-*i)ll/(a^ti)||  +  f  L|lxs-xti||  ds. 
Jti 


Now,  substituting  t  =  t  —  t\  and  o  —  s  —  t\,  this  becomes 

l|xT+ti  -Xfill  <  rll/(xti)||  +  f  L\\x<,+ti  -XtJl  dcr 

J  0 


Finally,  defining  u{t)  =  llxr+ti  —  Xti||,  this  becomes 

u{t)  <  Tl|/(xti)||  +  /  Lu(a)  da. 

Jo 

The  result  now  follows  firom  the  well-known  Bellman-Gronwall  inequality  [58,  p.  252].  □ 

Corollary  7.25  Under  the  same  assumptions  plus  the  fact  that  the  system  was  in  state  Xto 
at  time  to  <ti  <  t2, 

Ikt,  -  xtJI  <  \\f{xto)\\L-He^^'^-*^^  - 


Proof.  Note  that  Lipschitz  continuity  gives 


ll/(xtJl|<IK-Xto||-h]|/(xto)||. 
But,  the  lemma  gives  in  turn 

K  -  itoll  <  -  !)• 
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So  that  the  result  follows.  LJ 

The  following  lemma  is  well-known  (see,  e.g.,  [73,  p.  169]). 

Lemma  7.26  Let  y{t),z{t)  be  solutions  to  x{t)  =  f{x{t))  where  f  has  global  Lipschitz 
constant  L>0.  Then  for  allt>to, 

\\yit)-zit)\\  <  \\y{to)-z{to)\\e^^*-^<>\ 


□ 


Chapter  8 

Analysis  Tools 


§§8.2-8.4  introduce  some  analysis  tools  for  continuous  switching  systems.  We  prove  theo¬ 
rems  regarding  limit  cycles  and  robustness  of  such  systems.  The  remainder  of  the  chapter 
outlines  some  work  on  the  stability  analysis  of  switched  and  hybrid  systems.  We  introduce 
multiple  Lyapunov  functions  as  a  tool  for  analyzing  Lyapunov  stability  and  use  iterated 
function  systems  {IFS)  theory  as  a  tool  for  Lagrange  stability.  We  also  discuss  the  case 
where  the  switched  systems  are  indexed  by  an  arbitrary  compact  set. 

§8.1  INTRODUCTION 

In  the  first  part  of  the  chapter,  we  develop  general  tools  for  analyzing  continuous  switching 
systems.  For  instance,  we  prove  an  extension  of  Bendixson’s  Theorem  to  the  case  of  Lips- 
chitz  continuous  vector  fields.  This  gives  us  a  tool  for  analyzing  the  existence  of  limit  cycles 
of  continuous  switching  systems.  We  also  prove  a  lemma  dealing  with  the  continuity  of  dif¬ 
ferential  equations  with  respect  to  perturbations  that  preserve  a  linear  part.  Colloquially, 
this  lemma  demonstrates  the  robustness  of  ODEs  with  a  linear  part.  For  purpose  of  discus¬ 
sion,  we  call  it  the  Linear  Robustness  Lemma.  This  lemma  is  useful  in  easily  deriving  some 
of  the  common  robustness  results  of  nonlinear  ODE  theory  (as  given  in,  for  instance,  [11]). 
This  lemma  also  becomes  useful  in  studying  singular  perturbations  if  the  fast  dynamics  are 
such  that  they  maintain  the  corresponding  algebraic  equation  to  within  a  small  deviation. 
We  give  some  simple  propositions  that  allow  us  to  do  this  type  of  analysis. 

The  extension  of  Bendixson’s  Theorem  and  the  Linear  Robustness  Lemma  have  uses 
beyond  those  explicitly  espoused  here  and  should  be  of  general  interest  to  systems  theorists. 

§8.5  introduces  “multiple  Lyapunov  functions”  as  a  tool  for  analyzing  Lyapunov  stability 
of  switched  systems.  In  §8.6  iterative  function  systems  are  presented  as  a  tool  for  proving 
Lagrange  stability  and  positive  invariance.  We  also  address  the  case  where  {l,...,iV}  in 
Equations  (4.6)  and  (4.7)  is  replaced  by  an  arbitrary  compact  set.  We  conclude  with  some 
discussion. 

Appendix  A  collects  some  tedious  proofs.  Appendix  B  treats  the  background,  statement, 
and  proof  of  our  extension  of  Bendixson’s  Theorem. 

In  the  next  chapter,  we  use  the  above  tools  to  analyze  some  example  continuous  switch¬ 
ing  systems  motivated  by  a  realistic  aircraft  control  problem. 

§8.2  EXISTENCE  OF  LIMIT  CYCLES 

Suppose  we  are  interested  in  the  existence  of  limit  cycles  of  continuous  switching  systems 
in  the  plane.  The  traditional  tool  for  such  analysis  is  Bendixson’s  Theorem.  But  under  our 
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model,  systems  typically  admit  vector  fields  that  are  Lipschitz,  with  no  other  smoothriess 
assumptions.  Bendixson’s  Theorem,  as  it  is  traditionally  stated  (e.g.,  [67,  142]),  requires 
continuously  differentiable  vector  fields  and  is  thus  not  of  use  in  general.  Therefore,  we 
offer  an  extension  of  Bendixson’s  Theorem  to  the  more  general  case  of  Lipschitz  continuous 
vector  fields.  Its  proof  is  based  on  results  in  geometric  measure  theory  (which  are  discussed 
in  Appendix  8.10). 

Theorem  8.1  (Extension  of  Bendixson’s  Theorem)  Suppose  D  is  a  simply  connected 
domain  in  and  f{x)  is  a  Lipschitz  continuous  vector  field  on  D  such  that  the  quantity 
V/(x)  {the  divergence  of  f,  which  exists  almost  everywhere)  defined  by 

V/(x)  = 

is  not  zero  almost  everywhere  over  any  subregion  of  D  and  is  of  the  same  sign  almost 
everywhere  in  H.  Then  D  contains  no  closed  trajectories  of 

Xlit)  =  fl[xi{t),X2{t)],  (8-1) 

X2{t)  =  f2[xi{t),X2{t)].  (8.2) 

Proof.  Similar  to  that  of  Bendixson’s  Theorem  [142,  pp.  31-32]  after  using  an  extension 
of  the  divergence  theorem  known  as  the  Gauss-Green-Federer  Theorem  [109,  pp.  114-115]. 

(See  Appendix  8.10.)  •  •  r  P 

Finally,  we  give  an  example  which  shows  the  necessity  of  Lipschitz  continuity  of  the 

vector  fields. 

Example  8.2  Consider  Example  8.9.  Note  that  if  the  roles  of  A  and  B  are  interchanged, 
then  the  resulting  system  is  asymptotically  stable.  Thus,  continuity  of  solutions  and  the 
intermediate  value  theorem  imply  that  there  exists  A  €  (0, 1)  such  that  fi  —  \B  +  {1  —  A)  A 
and  /2  =  AA  +  (1  -  \)B  results  in  a  closed  trajectory.  Yet,  V/i  <  0  and  V/2  <  0. 

§8.3  ROBUSTNESS  OF  ODEs 

In  this  subsection,  we  summarize  some  results  that  show  the  robustness  of  solutions  of 
ordinary  differential  equations  with  respect  to  perturbations  of  the  vector  field.  First,  we 
give  and  prove  a  basic  lemma  in  ODE  theory  that  demonstrates  robustness  of  solutions  to 
arbitrary  perturbations.  Then,  we  consider  perturbations  that  preserve  a  linear  part.  This 
allows  us  to  obtain  more  useful  bounds.  We  call  the  result  the  Linear  Robustness  Lemina. 

The  proofs  of  both  lemmas  depend  critically  on  the  well-known  Bellman-Gronwall  in¬ 
equality  [58,  p.  252],  which  is  reprinted  in  Appendbc  8.9  for  convenience.  The  first  is  a  basic 
lemma  in  ODE  theory  that  was  given  without  proof  in  [153].  It  is  useful  as  a  comparison 
with  our  new  result,  Lemma  8.4.  For  completeness,  we  furnish  a  proof  in  Appendbc  8.9. 

Lemma  8.3  Given 

x  =  F{x,t),  1(0)  =xo, 
y  =  G{y,t),  y(0)  =  xo- 


^Proofs  that  interrupt  discussion  flow  or  do  not  use  novel  techniques  are  relegated  to  Appendix  8.9. 
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Suppose  that  F  is  globally  Lipschitz  continuous  and  “close  to  G,”  i.e., 

\\F{x,t)  -  F{y,t)\\<L\\x-y\l  for  all  x,y,t, 

|lF(x,f)  -  G(a;,i)il  <  for  all  x,t. 

Then  if  L^O 

lk(<)  -  y(*)ll  <  2  ~  0  ’ 

IfL  =  0,  then  |lx(i)  -  y(t)||  <  et 

Proof.  (See  Appendix  8.9.)  CH 

The  problem  with  this  result  is  that  (except  in  the  trivial  case)  L  >  0,  so  the  bound 
diverges  exponentially.  Thus  it  is  not  useful  in  deducing  stability  of  a  nearby  system, 
nor  in  examining  robustness  of  a  well-behaved  model  to  perturbations  in  the  vector  held. 
There  are  some  tools  for  this  in  the  literature,  under  the  heading  “stability  under  persistent 
disturbances.”  For  example,  [127,  p.  72]  gives  a  local  result.  We  are  more  interested  in 
what  one  can  say  globally.  Along  these  lines  we  consider  perturbations  that  preserve  a 
well-defined  portion  of  the  dynamics,  a  linear  part.  Here  is  our  main  result: 

Lemma  8.4  (Linear  Robustness  Lemma)  Given 

x  =  Ax  +  F{x,t),  x{0)  =  xo, 
y  =  Ay  +  G{y,  t),  y{0)  =  xq. 

Suppose  that  F  is  globally  Lipschitz  continuous  and  “close  to  G,”  i.e., 

||F(x,t) -F(y,t)||  <I|lx-yl|,  for  all  z,  y, i, 

||F(a:,t)  —  G(i,t)||  <  e,  for  all  x,t, 

Then  er  ,  \ 

£orall(>0 

when 

lies'll,  <  ce"*,  (8.3) 

where  ||  •  ||i  is  the  induced  norm  associated  with  the  norm  ||  •  [j  and  c  >  1,  t)  +  cL  0,  rj  0, 
and  L  >  0. 

Proof.  (See  Appendix  8.9.)  D 

Corollary  8.5  In  some  special  cases  not  covered  above  we  have: 

1.  If  L  =  0  but  T]  ^0,  then 


2.  //  7?  =  0  and  L  =  0,  then 

3.  Ifr]  =  0  but  L>  0,  then 


l|a:(<)  -y(t)||  <  cet. 
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4.  Jf  T}^0  and  L>0  but  7j  +  cL  =  0  (this  means  t)  <  0),  then 

i|a:(<)  -  J/(*)ll  <  -  e’'*]  • 

Proof.  (See  Appendix  8.9.)  O 

The  similarity  of  Lemmas  8.3  and  8.4  is  easy  to  see.  Their  proofs  are  also  similar.  The 
most  important  distinction  arises  when  A  is  stable  and  ri  can  be  chosen  negative.  Indeed, 
if  7?  +  cL  <  0,  then  we  can  guarantee  nondivergence  of  the  solutions. 

The  proof  can  easily  be  extended  to  the  case  where  A  is  time-varying: 

Corollary  8.6  Lemma  8.4  and  Corollary  8.5  hold  when  A  is  time  varying,  with  Equation 
(8.3)  replaced  by 

where  $(<,s)  is  the  transition  matrix  of  the  time-varying  linear  matrix  A{t). 

Proof.  Proof  is  the  same  as  that  for  Lemma  8.4,  replacing  by  $(t,  s).  □ 

Then,  the  case  L  =  0  subsumes  some  of  the  global  results  of  stability  under  persistent 

disturbances,  e.g.,  [11,  p.  167]. 

§8.4  SINGULAR  PERTURBATIONS 

The  standard  singular  perturbation  model  is  [87] 

X  =  f{x,z,€,t),  x{to)  =  XQ,  x€R^,  (8-4) 

ez  —  g{x,z,€,t),  z{to)  =  ZQ,  2  €  ,  (8.5) 

in  which  the  derivatives  of  some  of  the  states  are  multiplied  by  a  small  positive  scalar  e. 
When  we  set  e  =  0,  the  state-space  dimension  reduces  from  n  -I-  m  to  n  and  the  second 
differential  equation  degenerates  into  an  algebraic  equation.  Thus,  Equation  (8.4)  represents 
a  reduced-order  model,  with  the  resulting  parameter  perturbation  being  “singular.”  The 
reason  for  this  terminology  is  seen  when  we  divide  both  sides  of  Equation  (8.5)  by  e  and  let 
it  approach  zero. 

We  make  use  of  the  simpler  model 

X  =  f{x,z,t),  x{tQ)=xo,  x€R", 

z  =  a^[g{x,t)  -  z],  z{to)  =  zo,  2  €  R”* , 

where  we  have  e  =  l/a^,  a  a  nonzero  real  number.  With  this  rewriting,  one  sees  why 
Equation  (8.5)  is  said  to  represent  the  “fast  transients,”  or  fast  dynamics.  The  following 
lemma  shows  explicitly  a  certain  case  where  the  dynamics  can  be  made  so  fast  that  the 
resulting  “tracking  error”  between  u{t)  =  g{x{t),t)  and  z{t)  is  kept  small. 

Lemma  8.7  Let 

z{t)  =  a^{u{t)  -  z{t)), 

where  u  is  a  Lipschitz  continuous  (with  constant  L)  function  of  time.  Given  any  e  >  0,  if 
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we  can  choose  a  large  enough  so  that 

\z{t)  -  u{t)\  <  e,  t>0. 

Proof.  (See  Appendix  8.9.)  D 

The  result  can  be  extended  to  higher  dimensions  as  follows: 

Lemma  8.8  Let 

z{t)  =  a^A{u{t)  - 

where  u  and  z  are  elements  ofW  and  A  €  .  Further,  assume  that  A  is  positive  definite 

and  that  each  coordinate  of  u  is  a  Lipschitz  continuous  (with  constant  L)  function  of  time. 
Given  any  e  >  0,  if 

||2(0)  -  it(0)l|  =  Co  <  e, 

we  can  choose  a  large  enough  so  that 

Il2(t)-u(t)||<c,  t>0. 

Proof.  Similar  to  the  proof  of  Lemma  8.7.  [Hint:  consider  the  time  derivative  of  e^e, 
e  =  (z  —  u),  and  use  equivalence  of  norms  on  R”.]  D 

These  lemmas  allow  us  to  use  the  robustness  lemmas  of  the  previous  section  to  analyze 
certain  singular  perturbation  problems.  The  idea  of  the  preceding  lemmas  is  that  the  fast 
dynamics  are  such  that  they  maintain  the  corresponding  algebraic  equation,  z(t)  =u(t),  to 
within  a  small  deviation  (cf.  invariant  manifolds  [87,  p.  18]). 

§8.5  MULTIPLE  LYAPUNOV  FUNCTIONS 

In  this  section,  we  discuss  Lyapunov  stability  of  switched  systems  via  multiple  Lyapunov 
functions  (MLF)  The  idea  here  is  that  even  if  we  have  Lyapunov  functions  for  each  system 
fi  individually,  we  need  to  impose  restrictions  on  switching  to  guarantee  stability.  Indeed, 
it  is  easy  to  construct  examples  of  two  globally  exponentially  stable  systems  and  a  switching 
scheme  that  sends  all  trajectories  to  infinity: 

Example  8.9  Consider  fi{x)  =  Ax  and  f2{x)  =  Bx  where 

-0.1  10 
-1  -0.1  ■ 

Then  x  =  fi{x),  is  globally  exponentially  stable  for  i  =  1,2.  But  the  switched  system  using 
fi  in  the  second  and  fourth  quadrants  and  /a  in  the  first  and  third  quadrants  is  unstable.  See 
Figures  8-1-8-3,  which  plot  ten  seconds  of  trajectories  for  fi,  ft,  and  the  switched  system 
starting  from  (1,0),  (0,1),  (10“®,10~®),  respectively. 

We  assume  the  reader  is  famiUar  with  basic  Lyapunov  theory  (continuous  and  discrete 
time),  say,  at  the  level  of  [96].  The  level  of  rigor  of  the  proofs  is  similar  to  those  in  that 
book.  We  let  5(r),  B{r),  and  B(r)  represent  the  sphere,  ball,  and  closed  ball  of  Euclidean 
radius  r  about  the  origin  in  R" ,  respectively. 

Below,  we  defi  with  systems  that  switch  among  vector  fields  (resp.  difference  equations), 
over  time  or  regions  of  state-space.  One  can  associate  with  such  a  system  the  following 


-0.1  1 
10  -0.1 
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Figure  8-1:  Trajectory  of  /i. 


X 

Figure  8-2:  Trajectory  of  f2- 

(anchored)  switching  sequence,  indexed  by  an  initial  state,  xq'. 

The  sequence  may  or  may  not  be  infinite.  In  the  finite  case,  we  may  take  t^+i 
with  all  further  definitions  and  results  holding.  However,  we  present  in  the  sequel  only 
in  the  infinite  case  to  ease  notation.  The  switching  sequence,  along  with  Equation  (4.6), 
completely  describes  the  trajectory  of  the  system  according  to  the  following  rule:  {ik,tk) 
means  that  the  system  evolves  according  to  x{t)  =  /tt(a:(f),t)  for  tk<t<  tk+v  We  denote 
this  trajectory  by  a;s(-)-  Throughout,  we  assume  that  the  switching  sequence  is  minimal 

in  the  sense  that  ij  ij+i,  j  €  Z+.  _  _ 

We  can  take  projections  of  this  sequence  onto  its  first  and  second  coordinates,  yielding 

the  sequence  of  indices, 

7ri(5)=xo; 
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Figure  8-3:  Trajectory  of  switched  system. 


and  the  sequence  of  switching  times, 

^2(^)  ~  ^0)  ^0)  ^1)  •  •  •  )  •  •  •  > 

respectively.  Suppose  5  is  a  switching  sequence  as  in  Equation  (8.6).  We  denote  by  51^  the 
sequence  of  switching  times  whose  corresponding  index  is  i  for  the  discrete  case  and  the 
endpoints  of  the  times  that  system  i  is  active  in  the  continuous-time  case.  The  interval 
completion  I(T)  of  a  strictly  increasing  sequence  of  times  T  =  is  the  set 

U  (%,%+!)• 

jez+ 

Finally,  let  S{T)  denote  the  even  sequence  of  T: 

^2)  ^4)  •  •  •  • 

Below,  we  say  that  V  is  a  candidate  Lyapunov  function  if  V  is  a  continuous,  positive 
definite  function  (about  the  origin,  0)  with  continuous  partial  derivatives.  Note  this  assumes 
V (0)  =  0.  We  also  use 

Definition  8.10  Given  a  strictly  increasing  sequence  of  times  T  in  R  (resp.  Z),  we  say 
that  V  is  Lyapunov-like  for  function  f  and  trajectory  x{‘)  (resp.  x['])  over  T  if 

•  V(a:(t))  <  0  (resp.  V{x[t  +  1])  <  F(x[t]))  for  all  t  €  I{t)  (resp.  t  €  T), 

•  V  is  monotonically  nonincreasing  on  €{T)  (resp.  T). 

Theorem  8.11  Suppose  we  have  candidate  Lyapunov  functions  Vi,  i  =  1, . . , ,  N ,  and  vector 
fields  X  =  fi{x)  (resp.  difference  equations  x[k  +  1]  =  /i(a:[fc]))  vdth  fi{0)  =  0,  for  all  i.  Let 
S  be  the  set  of  all  switching  sequences  associated  with  the  system. 

If  for  each  S  €  S  we  have  that  for  all  i,  Vi  is  Lyapunov-like  for  and  xsi-)  over  Sji, 
then  the  system  is  stable  in  the  sense  of  Lyapunov. 

Proof.  In  each  case,  we  do  the  proofs  only  for  N  =  2. 
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•  Continuous- time:  Let  i?  >  0  be  arbitrary.  Let  Tni{oi)  denote  the  minimum  value 
of  Vi  on  5(a).  Pick  n  <  R  such  that  in  B(ri)  we  have  Vi  <  mi{R).  This  choice  is 
possible  via  the  continuity  of  Vi.  Let  r  =  min(rt).  With  this  choice,  if  we  start  in 
B{r),  either  vector  field  alone  will  stay  within  B{R). 

Now,  pick  pi  <r  such  that  in  B{pi)  we  have  Vi  <  mi{r).  Set  p  =  min(pi).  Thus, 
if  we  start  in  B{p),  either  vector  field  alone  will  stay  in  B{r).  Therefore,  wheriever 
the  other  is  first  switched  on  we  have  Vi{x{ti))  <  Tni{R),  so  that  we  will  stay  within 
BiR). 

•  Discrete-time:  Let  i?  >  0  be  arbitrary.  Let  mi{a,P)  denote  the  minimum  value  of 
Vi  on  the  closed  annulus  B(/?)  —  B (a).  Pick  Ro  <  R  so  that  none  of  the  fi  can  jump 
out  of  B{R)  in  one  step.  Pick  Vi  <  Ro  such  that  in  B(ri)  we  have  Vi  <mi{Ro,R)- 
This  choice  is  possible  via  the  continuity  of  V.  Let  r  =  min(ri).  With  this  choice,  if 
we  start  in  S(r),  either  equation  alone  will  stay  within  B{R). 

Pick  ro  <  r  so  that  none  of  the  fi  can  jump  out  of  B{r)  in  one  step.  Now,  pick  pi  <  ro 
such  that  in  B{pi)  we  have  Vi  <  mi(ro,r).  Set  p  =  min(pi).  Thus,  if  we  start  in  B{p), 
either  equation  alone  will  stay  in  5(ro),  and  hence  B{r).  Therefore,  whenever  the 
other  is  first  switched  on  we  have  Vi(x(ti))  <  mi{Ro,R),  so  that  we  will  stay  within 
B{Rq)i  and  hence  B{R). 

The  proofs  for  general  N  require  N  sets  of  concentric  circles  constructed  as  the  two  we^ 

in  each  case  above. 

Some  remarks  are  in  order: 

•  The  case  W  =  1  is  the  usual  theorem  for  Lyapunov  stability  [96].  Also,  compare 
Figures  8-4  and  8-5,  both  of  which  depict  the  continuous-time  case. 

•  The  theorem  also  holds  if  the  fi  are  time-varying. 

•  It  is  easy  to  see  that  the  theorem  does  not  hold  if  TV  =  oo,  and  we  leave  it  to  the 
reader  to  construct  examples. 


Figure  8-4;  Lyapunov  stability. 


§8.5  Multiple  Lyapunov  Functions 


127 


Figure  8-5:  Multiple  Lyapunov  stability,  N  =  2. 

Example  8.12  Pick  any  line  through  the  origin.  Going  back  to  Example  8.9  and  choosing 
to  use  fi  above  the  line  and  /2  below  it,  the  resulting  system  is  globally  asymptotically  stable. 
The  reason  is  that  each  system  is  strictly  stable  linear  and  hence  diminishes  Vi  =  x^PiX  for 
some  Pi  >  0.  However,  since  switchings  occur  on  a  line  through  the  origin,  we  are  assured 
that  on  switches  to  system  i,  Vi  is  lower  energy  than  when  it  was  last  switched  out. 

It  is  possible  to  use  different  conditions  on  the  Vi  to  ensure  stability.  For  instance, 
consider  the  following 

Definition  8.13  If  there  are  candidate  Lyapunov  functions  Vi  corresponding  to  fi  for  all 
i,  we  say  they  satisfy  the  sequence  nonincreasing  condition  for  a  trajectory  x(-)  if 

Vi^+,{x{tj+i))  <  V-^(x(tj)). 

This  is  a  stronger  notion  than  the  Lyapunov-like  condition  used  above. 

The  sequence  nonincreasing  condition  is  used  in  the  stability  (version  of  the  asymptotic 
stability)  theorem  of  [119].  Thus  that  theorem  is  a  special  case  of  the  continuous-time 
version  of  Theorem  8.11  above.  Moreover,  the  proof  of  asymptotic  stability  in  [119]  is 
flawed  since  it  only  proves  state  convergence  and  not  state  convergence  plus  stability,  as 
required.  It  can  be  fixed  using  our  theorem. 

Now,  consider  the  case  where  the  index  set  is  an  arbitrary  compact  set: 

X  =  f{x,  A),  Xe  K,  compact.  (8.7) 

Here,  x  €  M"  and  /  is  globally  Lipschitz  in  x,  continuous  in  A.  For  brevity,  we  only  consider 
the  continuous-time  case.  Again,  we  assume  finite  switches  in  finite  time. 

As  above,  we  may  define  a  switching  sequence 

S  —  Xq',  (Aq,  to)>  •  •  • )  ^  •  •  • 

with  its  associated  projection  sequences. 

Theorem  8.14  Euppose  we  have  candidate  Lyapunov  functions  Vx  =  V{-,X)  and  vector 
fields  as  in  Equation  (8.7)  with  /(0,A)  =  0,  for  each  X  e  K.  Also,  V  :  W  x  K  R+  is 
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continuous.  Let  S  he  the  set  of  all  switching  sequences  associated  with  the  system. 

If  for  each  S  €  S  we  have  that  for  all  i,  Vx  is  Lyapunov-like  function  for  fx  and  xs{-) 
over  SjA,  and  the  Vx  satisfy  the  sequence  nonincreasing  condition  for  xs{-),  then  the  system 
is  stable  in  the  sense  of  Lyapunov. 

Proof.  We  present  the  proof  in  the  case  that  K  is  sequentially  compact,  which  is  automatic 
if  X  is  a  metric  space.  The  general  case  follows  with  little  change  from  the  argument  below 
by  using  countable  compactness  and  nets  instead  of  sequences.  (See  [62, 113]  for  definitions). 

The  Lyapunov- like  and  sequence  nonincreasing  constraints  are  such  that  if  ‘Ki{S)  = 
xo;  Ao,  Ai,  A2, . . .,  then  the  state  x{t)  will  remain  within  the  set: 

Rv{xoM)  =  U  I  <  ''^(^o,Ao)}. 

Next,  note  that  if  xq  lies  in 

=  <  a:  I  sup  F(x,  A)  <  €  > , 

I  J 

then  the  state  will  remain  in  Re. 

Thus,  it  remains  to  show  that  given  any  e  >  0,  there  exist  e',  5  >  0  such  that 
B{S)  C  le'  n  B{e)  C  Re'  n  B(e)  C  B{e). 

Letting  m  denote  the  minimum  of  F  on  S{e)  x  K,  e'  =  m/2  satisfies  the  last  equation.  Now 
Ie>  contains  the  origin,  0  €  E".  Suppose  there  is  no  open  ball  about  0  in  le'.  Then  for  each 
n  €  Z+,  there  exists  yn  such  that 

Ilynll  <  l/«>  sup  F(y„,  A)  >  e'. 

Further,  we  may  take  each  of_the  y„  distinct.  Let  A„  €  if  be  the  point  at  which  the 
sup  above  is  attained.  Since  5(e')  x  K  is  sequentially  compact,  there  is  a  subsequence 
{iVik^K)}  converging  to  (0,A*)  with  V{yi„Xi^)  >  e',  a  contradiction  to  the  continuity  of 
V  and  the  assumption  that  V (0,  A)  =  0  for  all  A  €  if .  D 

This  theorem  is  a  different  generalization  of  the  aforementioned  theorem  in  [119]. 

§8.6  ITERATED  FUNCTION  SYSTEMS 

In  this  section,  we  study  iterated  function  systems  theory  as  a  tool  for  Lagrange  stability. 
We  begin  with  some  background  from  [12,  143,  48]: 

Definition  8.15  Recall  that  a  contractive  function  /  is  one  such  that  there  exists  s  <  1 
where  d(/(x),  f{y))  <  sd{x,  y),  for  all  x,y. 

An  iterated  function  system  or  IFS  is  a  complete  metric  space  and  a  set  {fi}iei 
of  contractive  functions  such  that  I  is  a  compact  space  and  the  map  (x,i)  i-t  fi{x)  is 
continuous. 

The  image  of  i,  compact  set  X  under  an  IFS  is  the  set  Y  =  Uig/  If  is  compact. 

Now  suppose  W  is  an  IFS.  Let  5(W)  be  the  semigroup  generated  by  W  under  composition. 
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For  example,  if  VF  =  {/,  g}  then 


S{W)=  f,g,f  o  fJog,go  f,gog,.... 

Now,  define  Aw  to  be  the  closure  of  the  fixed  points  of  S{W).  We  have 
Theorem  8.16  Suppose  W  =  is  an  IFS  on  X.  Then 

•  Aw  is  compact. 

•  Aw  =  [Jiei'^iiAw)- 


•  For  all  X  £  X, 

Aw  =  (J  { o  ’"trj  o  •  •  •  O  j  , 

a 

ly/iere  c  =  (cTi,  (72, . . .),  ft  € /. 

The  relevance  of  this  theorem  is  twofold: 

•  Aw  is  an  invariant  set  under  the  maps  {tOt}te/. 

•  All  points  approach  Aw  under  iterated  composition  of  the  maps  {iOi}te/. 

Clearly,  this  theory  can  be  applied  in  the  case  of  a  set  of  contractive  discrete  maps 
indexed  by  a  compact  set  (usually  finite).  Thus,  it  is  directly  applicable  to  systems  of  the 
form  Equation  (4.7). 

Example  8.17  The  following  IFS  is  well-known: 


Fiix)  = 


■  0.5 

0 

X  + 

0.5  • 

0 

0.5 

0.5  •  l{j<3} 

i  =  1,2,3. 


It  is  pictured  in  Figure  8-6. 

To  obtain  contractive  maps  while  switching  among  differential  equations  requires  a  little 
thought.  Assume  there  is  some  lower  limit  T  on  the  inter-switching  time.  Now,  notice  that 
for  any  inter-switching  time  r  >  T,  there  is  a  decomposition  into  smaller  intervals  as  follows: 

M 

r  =  53ti,  Ue[T,2T]. 

1=1 

Proof.  Let  k  =  [r/{2T)\  and  g  =  r  -  2Tk.  Now,  2T  >  g  >  0.  If  g  =  0,  the  decomposition 
is  t,  =  2T,  i  =  1, . . . ,  fc.  If  2r  >  g  >  T,  the  decomposition  is  tj  =  2T,  i  =  1, . . . ,  fc;  tk+i  =  g; 
the  first  equation  not  applying  if  fe  =  0.  Finally,  if  T  >  g  >  0,  then  (we  must  have  A:  >  1 
since  r  >  T)  and  2T  >  g -f  T  >  T,  so  the  decomposition  is  U  =  2T,  i  =  1, ...  ,k -  1-,  tk  =  T\ 
tk+i  =  r  -1-  g;  the  first  equation  not  applying  if  fe  =  1.  □ 

Therefore,  we  can  convert  switching  among  vector  fields  into  an  IFS  by  letting  I  = 
Ui=i  [^5  2r].  In  particular,  we  see  that  for  each  i,  if  it  is  active  for  a  time  r  >  T,  we 

can  write  the  solution  in  that  interval  as  <f>i.{x)  =  {o^=i(t>\.){x),  where  ^  is  the  fundamental 
solution  for  fi  acting  for  time  t.  Thus  the  switching  sequence  can  be  converted  to  an  iterated 
composition  of  maps  indexed  by  the  compact  set  I. 

The  other  interesting  point  about  IFS  theory  is  that  the  different  vector  fields  (or  differ¬ 
ence  equations)  need  not  have  the  same  equilibrium  point.  This  is  important  as  it  appears 
to  be  the  usual  case  in  switched  and  hybrid  systems  (cf.  Example  3.4). 
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Figure  8-6:  Example  IFS. 


Example  8.18  Starting  with  Example  8.17,  we  consider  a  corresponding  differential  IFS 
(DIFS),  with 


Fi{x)  = 


0 

X  + 

—a 

Ol  •  l{i<3}  _ 

i  =  1,2,3. 


It  is  pictured  in  Figure  8-7  for  a  =  ln2  and  T  —  2/3. 

In  conclusion,  in  IFS  we  have  a  tool  for  analyzing  the  Lagrange  stability  and  computing 
the  invariant  sets  of  switched  systems  of  the  form  Equations  (4.6)  and  (4.7).  The  resulting 
sets  Aw  are  reminiscent  of  those  for  usual  IFS  (see  [12]),  although  we  don’t  give  any  here. 
The  reader  may  consult  [12]  for  algorithms  to  compute  such  invariant  sets. 


§8.7  DISCUSSION 

In  both  the  MLF  and  IFS  cases,  the  stability  results  are  sufficiency  conditions  on  the 
continuous  dynamics  and  switching.  This  work  represents  the  rudiments  of  a  stability 
theory  of  the  systems  in  Equations  (4.6)  and  (4.7)  and,  in  turn,  of  hybrid  systems.  We  also 
discussed  the  case  where  {1, . . . ,  iV}  in  Equations  (4.6)  and  (4.7)  is  replaced  by  an  arbitrary 
compact  set. 

For  future  directions,  we  offer  the  following  brief  treatment.  In  searching  for  necessary 
and  sufficient  stability  criteria,  we  expect  that  the  theory  in  [94]  appears  helpful.  An  early 
use  of  our  MLF  theory  is  given  in  [147],  which  deals  with  convergence  of  a  combined  scheme 
for  robotic  planning  and  obstacle  avoidance.  As  far  as  IFS,  we  have  yet  to  explore  their  full 
potential.  For  instance,  we  can  state  IFS  theorems  analogous  to  Theorem  8.11,  namely,  m 
which  the  maps  need  only  be  contractions  on  the  points  (time  periods)  on  which  they  are 
applied.  Finally,  if  there  is  no  lower  limit  T  on  the  inter-switching  time,  then  we  are  not 
assured  to  have  a  contraction  mapping.  However,  as  long  as  we  have  only  finite  switches  in 
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Differential  IFS:  Sierpinksi  Triangle,  T*0.667  (10000  points) 


X 


Figure  8-7:  Example  DIFS. 


finite  time,  one  expects  that  the  trajectories  should  be  well-behaved  (e.g.,  invoke  continuity 
of  ODE  solutions  and  take  convex  hulls). 

Finally,  it  is  not  hard  to  generalize  our  MLF  theory  to  the  case  of  different  equilibria, 
which  is  generally  the  case  in  hybrid  systems.  For  example,  under  a  Lyapunov-like  switching 
rule,  after  all  controllers  have  been  switched  in  at  level  a,,  the  set  Uj  is  invariant. 


§8.8  NOTES 

The  work  in  §§8.2-8.4  first  appeared  in  [21],  later  summarized  in  [28].  The  work  in  §8.5 
and  §8.6  was  begun  in  [23]  and  continued  in  [30].  The  extension  of  Bendixson’s  Theorem 
and  the  Linear  Robustness  Lemma  have  applicability  beyond  the  systems  discussed  in  this 
thesis.  The  Linear  Robustness  Lemma  was  published  in  [29]. 

After  this  work  was  published,  we  became  aware  of  the  related  work  in  [118].  There 
Pavlidis  concludes  stability  of  differential  equations  containing  impulses  by  introducing  a 
positive  definite  function  which  decreases  during  the  occurrence  of  an  impulse  and  remains 
constant  or  decreases  during  the  “free  motion”  of  the  system.  Hence,  it  is  a  special  case  of 
our  results. 

In  personal  discussions,  Prof.  Wyatt  S.  Newman  essentially  conjectured  Theorem  8.11 
in  the  continuous-time  setting. 

§8.9  APPENDIX  A:  ASSORTED  PROOFS 

§8.9.1  CONTINUITY  LEMMAS 

The  proofs  of  our  continuity  lemmas  depend  critically  on  the  well-known  Bellman-Gronwall 
inequality  [58,  p.  252]: 

Lemma  8.19  (Bellman-Gronwall)  Let 
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1-  f>  9>  it;  IR+  — >■  K  and  locally  integrable; 


2.  g>0,  k>  0; 


3.  g  G  L'^ ; 


4.  gk  is  locally  integrable  on  IR+ . 


Under  these  conditions,  if  u  :  IR+  — ^  M  satisfies 

u(t)  <  f{t)  4-  g{t)  I  k{T)u{T)dT,  for  allteR+, 

Jo 

then 

u{t)  <  f{t)  +  g{t)  k{T)f{T)  l^exp  j  k{a)g{a)da  dr,  for  alltGR^. 
Proof,  [of  Lemma  8.3]  For  any  t>0, 

x(t)  =  xo+  [  F{x,T)dT, 

Jo 

y{t)  =  X0+  [  G{y,T)dT. 

Jo 

Subtracting  yields 

=  f  F{x,T)dT-  f  G{y,T)dT 
Jo  Jo 

=  Af(x,  t)  -  F{y,  t)  +  F{y,  r)  -  G{y,  T)]dT, 

Jo 

Mt)-y{t)\\  =  y\F{x,T)-F{y,T)  +  Fiy,T)-G{y,T)]dT 

<  t  \\F{x,T)-F{y,T)\\dT+  [  \\F{y,T)  -  G{y,T)\\dT 

Jo  Jo 

<  L  ['\\x{r)-y{T)\\dT  +  et 

Jo 

Using  the  Bellman-Gronwall  Lemma,  we  obtain 

Mt)  -  y{t)\\  <  et  +  Ler  Jexp  LdaJ  dr 


IfL  =  0 


Mt)-y{t)\\  =  et, 
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and  if  L  >  0,  we  compute 


re  ^'^dr  = 


-(-Lr-l) 


e  1 

—jy{Lt  +  l)  + 


Therefore, 


Proof,  [of  Lemma  8.4]  For  any  i  >  0, 


x(t)  =  e^‘xo+  /  '^)F(x,T)dT, 

Jo 

y{t)  =  e'^^xo  +  /  e'^(*“'^^G(2/,T)dT. 
Jo 


Subtracting  yields 


xit)-y{t)  =  [\^^^-^^[F{x,T)-F{y,r)  +  F{y,T)-G{y'T)]dT, 

Jo 

i|x(t)  -  y(t)||  <  j'  ||e^(*-^)l|i||F(x,r)  -  Fiy,T)\\dT  +  lle^(‘-^)|ii||F(y,T)  -  G{y,T)\\d^ 

<  if  \\e^^^~'"'>\\i\\x{T)-y{T)\\dT  +  e  f  ||e^(*"’’)||tdT. 

Jo  Jo 


Now  we  are  given  that  l|e"^®||i  <  ce^^,  so  that 


Therefore, 


<  ce^^fe-^dT 

Jo  Jo 

lli(t)  -  !/(()||  <  cie'"  e‘’'||i(T)  -  sWIliT  +  ^  (e’’*  -  l)  . 


Using  the  Bellman-Gronwall  Lemma,  we  obtain 


\\x{t)  -  y(«)||  <  ^  (e’'*  -  l)  +  cLe'^  {e^  -  1)  Jexp ^  e  ^^cLe^^'da  dr 

=■.  !£  Lvt  _  1  +  cLe^t  e^(t-r)^^j 
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ec 

1 

ec 

ri 

ec 

n 

ec 


-cLr  _  -(rj+cL) 


j  dr 


(8.10) 


e’J*  _  1  +  (--  — e-"^‘  +  - 

e  i  +  cije  ^77  +  cL  rj  +  cL/J 

e^t  _  1  +  g(,+cL)t  _  gr,t  ^  _  _^e(n+cL)t 


r)  +  cL  r}  +  cL 


cL 


-  1 


1‘- 


,(>?+cL)t 


T]  [r]  +  cL 

=  !£  [_^!?_1 

T)  [t}  +  cL\  L  -I 

=  ^.-—  -  ll . 

r/  +  cL  L  J 

Proof,  [of  Corollary  8.5]  Now  we  deal  some  special  cases  not  covered  above: 

1.  If  L  =  0  but  7?  7^  0,  then  Equation  (8.9)  gives 

Nt)-y(t)ll<|(e’’*-l). 

If  7/  =  0  then  Equation  (8.8)  is  replaced  by 

r  <  ct. 

Jo 

2.  So,  if  7/  =  0  and  L  =  0 

\\x{t)-y{t)\\<cet. 

3.  If  7?  =  0  and  L  >  0  then  Equation  (8.9)  is  replaced  by 

||x(t)  -  y{t)\\  <  cL  f  l|x(r)  -  y{T)\\dT  +  ect. 

J  0 

in  which  case  the  Bellman-Gronwall  Lemma  gives 

||x(t) -y(t)ll  <  ect  +  cL^  ecT  [expj^  cLda  dr 

<  ect  +  ec^L  f  Te‘^^(‘"'^^dT 

Jo 

<  ect  +  e(?Le‘^ 

Now  repeating  the  calculation  of  Equation  (8.8)  with  cL  identified  with  L, 


„2T^cLt  j\^cLrd^^ 

Jo 


□ 


(8.11) 


,2  T 


llx(f) -y(t)ll  <  ect  +  ecle' 


-cLt  1 

2.(I<t  +  1)  + 


{cir 


(cL)2 


^  ,  X  ^  I  ^  xXLt 

<  ect  —  ect  —  Y  +  yC 

^  iK-0- 
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4.  If  t;  ^  0  and  L  >  0  but  7)  +  cL  =  0  (this  means  ??  <  0),  then  Equation  (8.10)  and  the 
further  computations  simplify  to 


IkW-yWIl 


€C 

V 


+  cL 


ec 

V 


1  +  cL 


-  fe’'*  -  1  +  1  -  e-"^‘ 


—  e 


— cLt 


□ 


§8.9.2  SINGULAR  PERTURBATION  LEMMAS 

Proof,  [of  Lemma  8.7]  Let  e  —  z  —  u.  Then 


dt 


_  d'^z  d'^u 
dt  dt 

=  a^{u  —  z)  — 


d'^u 

dt 


d+u 


=  — a  e  — 


where 


d'^z 

— —  =  lim 
dt  h — ^0"^ 


dt  ' 
z{t  +  h)  —  z{t) 


Now,  since  u  is  Lipschitz,  we  have 


d+u 


dt 


<L. 


Thus,  if  we  choose  a  such  that  >  L/e,  then  when  e  >  e,  we  have 

d+e 


dt 


<  +  L  <  0. 


Similarly,  when  e  <  —6,  we  have 

>  Q^e  —  L  >  0. 

at 

Thus,  the  set  |e|  <  e  is  an  invariant  set.  □ 

§8.10  APPENDIX  B:  BENDIXSON  EXTENSION 

This  appendix  treats  the  background,  statement,  and  proof  of  our  extension  of  Bendixson’s 
Theorem. 

Bendbcson’s  theorem  gives  conditions  under  which  a  region  cannot  contain  a  periodic 
solution  (or  limit  cycle).  It  is  usually  stated  as  follows  (statement  and  footnote  adapted 
from  [142,  pp.  31-32]): 
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Theorem  8.20  (Bendixson’s  Theorem)  Suppose  D  is  a  simply  connected  domain  in 
r2  such  that  the  quantity  V/(x)  {the  divergence  of  f)  defined  by 

is  not  identically  zero  over  any  subregion  of  D  and  does  not  change  sign  in  D.  Then  D 
contains  no  closed  trajectories  of 

®i(<)  =  fi[xi{t),Mi)l 
i2{t)  =  f2[xi{t),X2{t)]. 

The  proof  of  Bendixson’s  Theorem  depends,  in  a  critical  way,  on  Green’s  Theorem.  The 
usual  statement. of  Green’s  Theorem  says  [109]  that  a  vector  field  fix)  on  a  compact 
region  A  in  R"  with  boundary  B  satisfies 

f  f{x)-n{A,x)d(T=  [  Vf{x)dC‘^x, 

Jb  JA 

where  n(A,  x)  is  the  exterior  unit  normal  to  A  at  x,  da  is  the  element  of  area  on  B,  and 
is  the  Lebesgue  measure  on  R".  It  is  possible,  however,  to  treat  more  general  regions 
and  vector  fields  that  are  merely  Lipschitz  continuous.  A  general  extension  is  the  so-caUed 
Gauss-Green-Federer  Theorem  given  in  [109].  Even  the  statement  of  this  theorem  requires 
the  development  of  a  bit  of  the  language  of  geometric  measure  theory.  We  state  a  relaxed 
version  of  this  theorem  that  is  still  suitable  for  our  purposes.  In  the  final  formula,  V/ 
exists  almost  everywhere  because  a  Lipschitz  continuous  function  is  differentiable  almost 

everywhere. 

Theorem  8.21  (Relaxation  of  Gauss-Green-Federer)  Let  A  be  a  compact  region  of 
R”  with  boundary  B.  Then  for  any  Lipschitz  vector  field  f{x), 

fix)  ■  n(A,  x)da  =  V/(x)d£”x. 

Now  we  C2in  prove  our  version  of  Bendixson  s  Theorem. 

Proof,  [of  Theorem  8.1]  The  proof  is  similar  to  that  of  Bendixson’s  Theorem  in  [142, 
pp  31  ^^] 

Suppose,  for  contradiction,  that  J  is  a  closed  trajectory  of  Equations  (8.1)  and  (8.2). 
Then  at  each  point  x  €  J,  the  vector  field  fix)  is  tangent  to  J.  Then  fix)  ■  n(5,x)  =  0  for 
all  X  €  J,  where  S  is  the  area  enclosed  by  J.  But  by  Theorem  8.21 

0  =  y  fix)  -niA,x)dl  =  jvfix)dC?x. 

Therefore,  we  must  have  either  (i)  V/(x)  is  zero  almost  everywhere,  or  (ii)  the  sets  {x  € 
5|V/(x)  <  0}  and  {x  €  S'|V/(x)  >  0}  both  have  positive  measure.  But  if  5  is  a  subset  of 
D\  neither  can  happen.  Hence,  D  contains  no  closed  trajectories  of  Equations  (8.1)-(8.2^ 


2A  connected  region  can  be  thought  of  as  a  set  that  is  in  one  piece,  i.e.,  one  in  which  every  two  points 
in  the  set  can  be  connected  by  a  curve  lying  entirely  within  the  set.  A  set  is  simply  connected  if  (1)  i  is 
connected  and  (2)  its  boundary  is  connected. 


Chapter  9 

Analyzing  Examples 


In  this  chapter,  the  attention  focuses  on  example  systems  and  their  analysis.  We  first 
analyze  a  class  of  two-state,  continuous  switched  systems,  proving  global  stability.  Using 
tools  from  §8,  we  conclude  stability  of  a  class  of  continuations  of  those  systems.  The  example 
systems  arise  from  a  realistic  aircraft  control  problem. 


§9.1  INTRODUCTION 

In  this  chapter,  we  use  the  tools  of  §8  to  analyze  some  example  continuous  switched  systems 
motivated  by  a  realistic  aircraft  control  problem.  In  the  next  section,  we  present  an  example 
continuous  switched  control  problem.  This  system  is  inspired  from  one  used  in  the  longi¬ 
tudinal  control  of  modern  aircraft  such  as  the  F-8  [134].  The  control  law  uses  a  “logical” 
function  (max)  to  pick  between  one  of  two  stable  controllers:  the  first  a  servo  that  tracks 
pilot  inputs,  the  second  a  regulator  about  a  fixed  angle  of  attack.  The  desired  effect  of  the 
total  controller  is  to  “track  pilot  inputs  except  when  those  inputs  would  cause  the  aircraft 
to  exceed  a  maximum  angle  of  attack.”  We  analyze  the  stability  of  this  hybrid  system  in 
the  case  where  the  pilot  input  is  zero  and  the  controllers  are  linear  full-state  feedback.  We 
call  this  the  max  system.  While  the  restriction  to  this  case  seems  strong,  one  should  note 
that  the  stability  of  such  systems  is  typically  verified  only  by  extensive  simulation  [134].  In 
this  chapter,  we  use  the  tools  discussed  above  to  prove  nontrivial  statements  about  the  con¬ 
troller’s  behavior.  For  example,  we  show  that  no  limit  cycles  exist  by  applying  our  extension 
of  Bendixson’s  Theorem.  We  also  show  that  the  family  of  linear  full-state  feedback  max 
systems  can  be  reduced  to  a  simpler  family  via  a  change  of  basis  and  analysis  of  equilibria. 
Finally,  we  give  a  Lyapunov  function  that  proves  that  all  systems  of  this  canonical  form  are 
globally  asymptotically  stable.  The  Lyapunov  function  itself  has  a  logical  component,  and 
the  proof  that  it  diminishes  along  trajectories  is  split  into  logical  cases. 

In  §9.3  we  analyze  a  “continuation”  of  the  max  system.  Specifically,  we  use  a  dynamic 
variable  (output  of  a  differential  equation)  instead  of  the  output  given  by  the  max  func¬ 
tion  directly.  This  corresponds  to  a  dynamical  smoothing  or  switching  hysteresis,  as  we 
motivated  above.  By  using  our  lemma  on  the  robustness  of  linear  ODEs,  we  conclude  sta¬ 
bility  properties  of  this  (singular  perturbation)  continuation  firom  those  of  the  original  max 
system. 

The  Appendix  collects  the  more  tedious  proofs. 
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§9.2  EXAMPLE  1:  MAX  SYSTEM 


As  an  example  of  a  switched  system,  we  consider  a  problem  combining  logic  in  a  continuous 
control  system.  Specifically,  we  start  with  the  system 


or,  symbolically, 


£ 

dt 


Q 

a 

a 

riz 


X 

a 

riz 


Ax  + 

Cix  +  Di5  =  Cix, 

C2X  +  D25, 


= 

-1  -10 

1  -1 

Q 

a 

+ 

-1 

0.1 

= 

’  0  1  ' 
0  -300 

Q 

a 

+ 

'  0  ' 
30 

S. 


S. 


These  equations  arise  fi-om  the  longitudinal  dynamics  of  an  aircraft  (see  Figure  9-1)  with 
reasonable  values  chosen  for  the  physical  parameters.  The  variable  6  is  the  pitch  angle  and 
Q  is  the  angle  of  attack.  The  input  command  S  is  the  angle  of  the  elevator.  The  normal 
acceleration,  n^,  is  the  output  variable  which  we  would  like  to  track,  i.e.,  we  assume  that 
the  pilot  requests  desired  values  of  riz  with  his  control  stick.  As  a  constraint,  the  output 
variable  a  must  have  a  value  not  much  larger  than  aum  (for  the  model  to  be  valid  and 
the  plane  to  be  controlled  adequately).  A  successful  controller  would  satisfy  both  of  these 
objectives  simultaneously  to  the  extent  that  this  is  possible;  we  desire  good  tracking  of  the 
pilot’s  input  without  violating  the  constraint  o;  <  aum  -H  e,  for  e  >  0  some  safety  margin. 


✓ 

✓ 

✓ 


Figure  9-1:  Longitudinal  Aircraft  View. 

Now,  suppose  that  two  controllers,  Ki  8ind  K2,  have  been  designed  to  output  and  S2 
such  that  (1)  E  is  regulated  about  a  =  aiim  when  5  =  <5i;  and  (2)  S  tracks  command  r  the 
pilot’s  desired  n^— when  6  -  S2,  respectively.  Finally,  suppose  that  we  add  the  following 
logical  block:  S  =  max(<5i,  ^2).  Control  configurations  much  like  this  (see  Figure  9-2)  have 
been  used  to  satisfy  the  objectives  of  our  aircraft  control  problem  [134]. 

To  our  knowledge,  the  stability  of  such  systems  has  only  been  probed  via  extensive 
simulation  [134].  In  the  remainder  of  this  section,  we  examine  the  stability  of  certain  cases 
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Figure  9-2;  The  Max  Control  System. 


of  this  control  system.  First,  we  limit  ourselves  to  the  case  where  both  controllers  are 
implemented  with  full-state  feedback.  We  discuss  the  well-posedness  of  the  system  and 
show  an  example  output  run.  Next,  we  consider  the  equilibrium  points  of  the  system  and 
their  stability  in  the  case  where  the  pilot’s  reference  input  (desired  normal  acceleration)  is 
clamped  to  zero.  More  practically,  we  answer  the  question,  What  is  the  behavior  of  this 
control  system  if  the  pilot  lets  go  of  the  control  stick? 

§9.2.1  PRELIMINARY  ANALYSIS  OF  THE  EXAMPLE  SYSTEM 

First  note  that  in  our  example  system,  the  pair  {A,B)  is  controllable.  To  make  some 
headway,  we  restrict  ourselves  to  the  special  case  where  the  controllers  Ki  and  K2  take  the 
form  of  full-state  feedback  plus  an  offset  term  (for  nonzero  outputs): 

=  —Fx  +  [Ci{—A  +  BF)  ^B]  ^oiiini! 

62  =  -Gx  +  [{C2-D2G){-A  +  BG)-^B  +  D2]~^r. 

For  convenience,  we  let 

ki  =  [Ci{-A  +  BFr^B]-\ 

k2  =  \{C2  ~  D2G){— A  +  BG)  ^B  +  D2] 

Such  constants  generally  need  not  exist.  However,  for  our  system  we  are  keenly  interested 
in  the  existence  of  We  have  the  following 

Fact  9.1  The  constant  ki  is  guaranteed  to  exist  for  our  system  whenever  F  is  chosen  such 
that  {A  —  BF)  is  stable? 

.  Proof.  (See  Appendix  9.5.)  D 


^We  assume  fc2  exists  since  it  does  not  affect  our  later  analysis,  which  is  in  the  case  r  =  0. 
*We  say  a  matrix  is  stable  when  all  its  eigenvalues  are  strictly  in  the  left-half  plane. 
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Thus,  the  resulting  max  control  law  exists.  It  is  simply 

<5  =  max(— Fx  +  fciaiim,  — Gx  +  A:2r).  (9-1) 

To  get  a  feel  for  how  the  example  system  behaves,  we  have  included  some  simulations. 
Figure  9-3  shows  an  example  run  of  the  just  the  tracking  portion  of  the  control  system 
(o^iim  —  0.6,  F  chosen  to  place  the  closed- loop  poles  at  —6  and  —7).  Part  (a)  shows  normal 
acceleration  tracks  the  desired  trajectory  well;  (b)  shows  the  aum  constraint  is  violated  to 
achieve  this  tracking. 

Figure  9-4  shows  the  outputs  when  the  full  max  control  system  is  activated  (with  both 
F  =  G  chosen  as  F  above).  One  easily  sees  that  the  controller  acts  as  expected:  it  tracks 
the  desired  command  well,  except  in  that  portion  where  tracking  the  command  requires 
that  the  aum  constraint  be  violated.  In  this  portion,  a  is  kept  close  to  its  constraint  value 
(the  maximum  value  of  a  in  this  simulation  run  was  0.6092). 

§9.2.2  ANALYSIS  FOR  THE  CASE  F  =  0 

The  first  thing  we  do  is  examine  stability  of  S  using  the  max  control  law  in  the  case  where 
r  =  0. 

NOTE.  Similar  analysis  holds  for  r  any  constant  after  change  of  variables.  If  r  is  given  by  an 
asymptotically  stable  differential  equation,  then  a  theorem  of  [142]  for  triangular  systems 
may  be  used  with  our  result  to  conclude  global  asymptotic  stability. 

In  the  case  r  =  0,  the  closed-loop  system  equations  are  then 

X  =  Ax-t-Bmax(-Fx-h  fciaiim, -Gx) 

=  (A  —  FG)x -h  J5  max((G  —  F)x -i- fciaiimjO). 

In  our  analysis  below,  we  suppose  that  we  have  done  a  reasonable  job  in  designing  the 
feedback  controls  F  and  G.  That  is,  we  assume  (A  -  BF)  and  (A  -  BG)  are  stable.  This 
is  possible  because  (A,  B)  controllable. 

Now,  recall  that  (A,  B)  controllable  implies  that  (A  —  BG,  B)  is  controllable.  Thus,  it 
suffices  to  analyze  the  following. 

Definition  9.2  The  max  system  is  defined  by 

Smax:  z  Az  +  Bmax{Fz +  ‘y,0), 

where  A  and  A  +  BF  are  stable  and  (A,  B)  is  controllable  and  7  =  feiaum. 

To  fix  ideas,  let’s  look  at  simulation  results.  Figure  9-5  shows  a  max  system  trajectory  with 

f=[-9  0],  7  =  -l. 

Figure  9-7^  shows  the  trajectory  resulting  from  the  same  initial  condition  for  the  system 
X  =  Ax;  Figure  9-8  for  the  system  i  =  (A  4*  BF)x.  Both  component  systems  are  stable. 

^We  intentionally  skipped  a  figure  to  allow  better  figure  placement  for  comparison. 
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To  simplify  analysis  of  the  max  system,  we  can  make  a  change  of  basis  (x  =  Pz),  yielding 
X  =  Pz 

=  PAz  +  PB  max(Fz  +  7, 0) 

=  PAP~^x  +  PB  max{FP~^x  +  7»  0)) 


where  P  is  any  nonsingular  matrix.  In  particular,  P  can  be  chosen  so  that  the  matrices 
PAP~^  and  PB  are  in  the  so-called  controller  canonical  form: 


PAP-^  = 
PB  = 


0  1 
— Oo  — Ol 

0  ' 

1  ■ 


(9.2) 

(9.3) 


Note  that  Oi  >  0  since  PAP~^  is  a  stable  matrix.  Renaming  matrices  again,  we  have 
Fact  9.3  The  max  system  Smax  can  be  reduced  to  the  system: 

X  =  Ax  +  B  max{Fx  +  7, 0), 

where  A  and  A  +  BF  are  stable,  {A,  B)  is  controllable,  and  the  matrices  A  and  B  are  in 
controller  canonical  form. 

We  can  do  one  more  thing  to  simplify  the  max  system  just  derived:  expand  the  equa¬ 
tions  using  the  fact  that  A  and  B  have  controller  canonical  form.  Doing  this — and  some 
equilibrium  point  analysis — we  obtain 

Remark  9.4  The  max  system  can  be  reduced  to  the  following  canonical  form,  denoted 
canonical  maoc  system: 


1. 


X  =  y, 

y  =  -ax-by+max{fx  +  gy  +  ’y,0), 

where  a,  b,  a  —  f,  and  b  —  g  are  greater  than  zero. 

2.  Further,  without  loss  of  generality,  we  may  assume  that  7  <  0,  m  which  case  the  only 
equilibrium  point  of  this  system  is  the  origin. 

Proof.  The  first  part  is  a  straightforward  calculation,  with  the  inequalities  on  the  constants 
arising  from  the  assumed  stability  of  A  and  A  -1-  BF. 

Now,  let’s  analyze  the  equilibrium  points  of  the  canonical  max  system.  The  relevant 
equations  are  y  =  0  and  ax  =  max{fx  -f  7,0).  This  second  one  must  be  analyzed  in  two 
cases: 


ax  =  0,  fx  +  ‘y  <0, 
ax  =  fx  +  'y,  /x  4-  7  >  0. 

Thus,  (0,0)  is  an  equilibrium  point  if  and  only  if  7  <  0;  (7/(0  -  /),0)  is  an  equilibrium 
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point  if  and  only  if 


J_ 

a  — 


y  7  +  7 


a 

a-f 


7 


7 


>  0, 

>  0, 
>  0, 


where  the  last  line  follows  from  a  and  a  —  f  greater  than  zero.  Therefore,  the  canonical 
max  system  has  exactly  one  equilibrium  point. 

Finally,  if  7  >  0,  changing  coordinates  to  2  =  x  -  7/(0  -  /)  yields  z  =  y  and 


=  -a(^z-\- -by  +  max  +gy  + 1,0^ 


=  —az  —  by  — 


aj 

a-f 


+  max  \fz  +  gy  + 


=  -az-by  +  fz  +  gy  +  max(o,-fz-gy--^^^ 

=  -{a- f)z-{b- g)y  + max  (^{-f)z  +  {-g)y+  >0^  • 

Now  introducing  new  variables  for  the  constants  in  parentheses,  we  obtain 


z  =  y, 

y  =  -az  —  by  +  max{fz  +  py  +  7>  0)- 

It  is  easy  to  check  that  the  new  variables  satisfy  the  inequalities  of  the  canonical  form. 
Further,  we  have  7  <  0,  and  thus  (0,0)  the  only  equilibrium.  □ 

Next,  note  that  this  is  equivalent  to  the  second-order  system: 

X  =  -ax  -bx  +  max{fx  +  gx  +  'r,  0), 


which  we  use  below. 

We  have  the  following  global  results  for  the  max  system  in  the  case  where  the  reierence 
input,  r,  is  zero: 

1.  Limit  cycles  don’t  exist.  Our  max  system  consists  of  a  logical  (though  Lipschitz 
continuous)  switching  between  two  stable  linear  systems,  both  of  which  admit  negative 
divergence  in  their  respective  regions.  Therefore,  by  Theorem  8.1,  no  limit  cycles  can 

exist. 

2.  The  system  is  globally  asymptotically  stable.  The  proof  is  detailed  below. 

To  prove  global  asymptotic  stability,  we  first  show 

Remark  9.5  The  following  is  a  Lyapunov  function  for  the  canonical  max  system: 

=  ii^+  f  [a^  -  max(/^ -I- 7, 0)]d^ 

2  Jo 

=  [  c(0<^^- 

2  Jo 


V 
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Proof.  The  proof  has  two  major  parts:  (i)  V  is  a  positive  definite  (p.d.)  function,  and 
(ii)  V<0. 

(i)  To  show  that  F  is  a  p.d.  function,  it  is  enough  to  show  that  xc(x)  >  0  when  x  ^  0 
and  c(0)  =  0.  The  second  fact  follows  from  7  <  0.  Computing 

xc(x)  =  ax^  -  X  max(fx  +  7, 0) 

_  f  ax^,  fx  +  j  <0, 

~  1  ax^  -  fx^  -  7X,  /x  +  7  >  0. 

That  the  desired  condition  holds  in  the  first  case  follows  immediately  from  a  >  0.  For 
the  second  case,  we  consider 

1.  X  >  0: 

ax^  -  /x^  -  7X  =  (a  -  /)x^  +  (-7)x  >0  +  0  =  0, 

2.  X  <  0: 

ax^  —  fx^  —  7X  =  ax^  +  (— x)(/x  +  7)  >  0  +  0  =  0. 


Thus  y  is  a  p.d.  function. 

(ii)  Next,  we  wish  to  show  that  F  <  0.  To  that  end,  we  compute 
V  =  XX  +  c(x)x 

=  x[-ax  -bx  +  max(/x  +  ^x  +  7, 0)]  +  axx  -  max(/x  +  7, 0)x 
=  -bx^  +  X  max(/x  +  px  +  7, 0)  -  x  max(/x  +  7,0). 

Now,  there  are  four  cases  to  be  dealt  with: 

1.  If  /x  +  5X  +  7  <  0  and  /x  +  7  <  0,  then  V  =  —bx^  <  0. 

2.  If  /x  +  px  +  7  >  0  and  /x  +  7  >  0,  then  V  =  -(6  -  g)x^  <  0. 

3.  If  /x  +  px  +  7  <  0  and  /x  +  7  >  0,  then 

V  —  —bx^  —  x(/x  +  7). 

If  X  >  0,  then  y  <  0.  If  x  <  0,  then,  using  {b  —  g)  >  0,  we  obtain 


b 

> 

9 

bx 

< 

pi> 

/x  +  7  +  6x 

< 

/x  +  7  +  px 

/x  +  7  +  6x 

< 

0, 

— x[/x  +  7  +  ftir] 

< 

0, 

y 

< 

0. 

4.  If  /x  +  px  +  7  >  0  and  /x  +  7  <  0,  then 

y  =  —bx^  +  x(/x  +  xp  +  7). 
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If  x  <  0,  then  V  <  0.  If  x  >  0, 

V  =  —(6  —  g)3p  +  x(/x  +  7)  <  0. 


□ 

Global  asymptotic  stability  results  from  the  facts  that  (1)  the  origin  is  the  only  invariant 
set  for  which  F  =  0  and  (2)  F(x)  00  as  ||xil  -t  00  [132]. 


§9.3  EXAMPLE  2:  THE  MAX  SYSTEM  CONTINUATION 

In  this  section  we  analyze  a  variant  of  the  max  system  introduced  in  §9.2.  Specifically,  recall 
that  the  max  system  can  be  reduced  to  the  canonical  form  of  Remark  9.4. 


i  =  y, 

y  =  -ax-by  +  max{fx  +  gy  +  'y,0), 

where  a,b,  a-  f,  and  6  -  5  are  greater  than  zero  and  7  <  0.  It  was  shown  in  %9.2  that  the 
only  equilibrium  point  of  this  system  is  the  origin,  which  is  globally  asymptotically  stable. 

We  now  examine  a  continuation  of  the  max  system  that  uses  a  differential  equation  to 
“dynamically  smooth”  the  max  function. 

Definition  9.6  The  max  system  continuation  is  defined  by 

X  =  Ax  +  BS, 

6  =  a^[max(Fx  +  7)  0)  “ 

where  A  and  A  +  BF  are  stable  and  {A,B)  is  controllable.  Also,  7  =  haum  and  a  #  0. 

This  equation  represents  a  smoothing  of  the  max  function’s  output;  it  provides  a  type  of 
dynamic  hysteresis  that  smooths  transitions.  Note  also  that  this  equation  represents  a 
singular  perturbation  of  the  original  max  system.  It  can  be  used  to  model  the  fact  that  the 
elevator  angle  does  not  exactly  track  the  desired  control  trajectory  specified  by  the  max 
function.  To  compare  the  max  and  max  continuation  systems,  consider  Figure  9-6.  This 
figure  shows  the  continuation  of  the  max  system  trajectory  of  Figure  9-5  with  a  =  16  and 
«5(0)  =  max(Fx(0) -1-7,0).  Note  that,  compared  with  the  original  max  system,  the  switching 
is  “delayed”  and  the  trajectories  are  smoother,  as  expected. 

By  changing  basis  with  the  matrix  T  =  blockdiag{F,  1},  where  P  is  chosen  so  that 
the  matrices  PAP-^  and  PB  are  in  the  so-called  controller  canonical  form  (see  Equations 
(9.2)  and  (9.3)),  we  obtain 

Definition  9.7  The  max  system  continuation  can  be  reduced  to  the  following  canonical 
form,  denoted  canonical  max  system  continuation; 

X  =  y, 

y  =  —ax  —  by +  5, 

6  -  a^[max(/x-t-yy  +  7v0) 

subject  to  initial  conditions 

x(0)  =  xo,  y(0)  =  yo,  <^(0)  =  max(/xo  +  gyo  +  7, 0), 
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where  a,  b,  a  —  f,  and  b  —  g,  are  greater  than  zero;  7  <  0;  and  a  ^  0. 

This  is  the  system  we  study  in  the  remainder  of  this  section.  Note  the  added  constraint  on 
the  initial  condition  of  5. 

NOTE.  The  constraint  on  <J(0)  is  for  convenience.  It  can  be  relaxed  to  be  within  e  of  this 
value  (where  e  arises  in  our  proofs  below),  with  the  same  analytical  results  holding  true. 
Specifically,  Fact  9.9  still  holds  when  Equation  (9.4)  is  replaced  by  |(J(0)  -  max(/a:(0)  + 
5y(0)  +  7,0)1  <  e. 


Remark  9.8  The  only  equilibrium  point  of  this  system  is  the  origin,  which  is  locally  asymp¬ 
totically  stable  {when  7  <  0). 


Proof.  From  the  first  two  equations,  we  have  the  constraints  y  =  0  and  5  =  ax.  From  the 
last  one  we  obtain  the  following  two  cases: 


1.  /x  +  7  <  0:  —ax  =  0,  which  implies  x  =  5  =  0. 

2.  /x  +  7  >  0:  (—a  +  f)x  +  7  =  0,  which  implies  x  =  j/ {a  —  /).  However,  this  can’t 


occur  since 


/x  +  7  = 


h 

ia-f) 


+  7  = 


ay 

J^) 


<0. 


The  origin  is  locally  asymptotically  stable  because  it  is  a  linear  system  in  some  neighborhood 
of  the  origin  (since  7  <  0).  d 

This  system  is  globally  asymptotically  stable  when  f  =  g  =  0,  because  it  reduces  to 
a  stable  linear  system  in  this  case.  For  the  special  case  where  7  =  0,  both  component 
linear  systems  can  be  made  stable  by  choosing  a  large  enough.  However,  this  in  itself  does 
not  imply  that  the  whole  system  is  stable.  We  say  more  about  this  case  at  the  end  of  the 
section. 

The  rest  of  this  section  explores  the  stability  of  the  max  system  continuation  by  using 
Lemma  8.4. 


§9.3.1  ASYMPTOTIC  STABILITY  WITHIN  ARBITRARY  COMPACT  SETS 

In  this  subsection  we  show  that  the  max  system  continuation  can  be  made  asymptotically 
stable  to  the  origin — within  an  arbitrary  compact  set  containing  the  origin — by  choosing 
the  parameter  a  Ijirge  enough. 

Important  Note.  Since  S  is  subject  to  initial  conditions  depending  on  x  and  y  (see 
Definition  9.6),  this  stability  is  with  respect  to  arbitrary  sets  of  initial  conditions 
for  X  and  y  only. 

This  subsection  only  considers  the  case  7  <  0.  In  this  case,  the  plane  /x  +  yy  +  7  =  0 
is  a  positive  distance,  call  it  d,  away  from  the  origin.  Further,  the  three-dimensional  linear 
system  associated  with  the  max  system  continuation  about  the  origin  with  matrix 

■  0  1  0  ■ 

—a  —b  1 
0  0  —a^ 
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is  asymptotically  stable,  so  there  is  some  (open)  ball  around  the  origin  (in  E^)  of  radius 
As<d  such  that  once  a  trajectory  of  the  max  system  continuation  enters  the  ball  of  radius 
As,  it  tends  toward  the  origin.  Similarly,  the  max  system  is  an  asymptotically  stable  linear 
system  near  the  origin,  so  there  is  some  ball  around  the  origin  (in  E^)  of  radius  Am  <  d 
such  that  once  a  solution  to  the  max  system  enters  the  ball  of  radius  Am,  it  tends  towards 
the  origin.  For  convenience,  define  A  =  min(Am5  A,). 

Now,  note  that  the  max  system  and  max  system  continuation  can  be  written  in  the 
form  required  by  Lemma  8.4  by  choosing 


i.e., 

F{x,y,t)  =  max{fx  +  gy  +  j,0), 

G{x,  y,  t)  =  6, 

where  S  =  a^[max(/x  +  gy  +  '1,0)  -  6],  An  important  fact  is  the  following; 

Fact  9,9  Given 

5(0)  =  max(/x(0)  +  gy{0)  +  7, 0)  (9.4) 

and  e  >  Of  we  can  choose  a  large  enough  so  that 

|(5(t)  -  max(/x(t)  +  gyit)  +  7, 0)|  <  e,  t  >  0. 

Proof.  Since  max(/x  +  gy  +  i,  0)  is  Lipschitz  continuous,  we  can  apply  Lemma  8.7  with 

u(-)  =  max(/x(0+5y(-)  +  T,O),  x(-)  =<J(-),  andeo  =  0.  _  _  □ 

Below,  let  //(<)  and  cr(t)  represent  solutions  to  the  max  and  max  continuation  systems, 
respectively.  Next,  consider  the  projection  operator 

7r:E^  ->•  E^, 

7r([x,y,^]^)  =  [x,yf  ■ 

Remark  9.10  lf^<0,  the  max  system  continuation  can  be  made  asymptotically  stable  to 
the  origin  within  an  arbitrary  compact  set  containing  it  by  choosing  the  parameter  a  large 
enough. 


Proof.  First,  pick  any  compact  set,  fi,  containing  the  origin  (of  the  max  system).  Next, 
we  examine  the  trajectories  of  the  max  and  max  continuation  systems  from  an  arbitrary 
initial  condition,  no  €  fi.  Recall  that  (Jq,  and  hence  ao,  is  completely  determined  by  no-  In 

particular,  (5o  =  max(/xo  +  gyo  +  7)  0)  and  7r(ao)  =  no- 

Since  the  max  system  is  globally  asymptotically  stable,  there  is  a  time,  such 

that  for  t  >  T,  we  have  \\n{t)\\  <  A/3.  Thus,  we  have  max(./x(t)  +  gy{t)  +7,0)  =  0  for  all 
t>T,  Now,  according  to  Fact  9.9  we  can  pick  a  large  enough  so  that 


c  <  min 


/  A  A(?7  +  cL)  \ 
(s’’  3c(e(’»+^^)r  -  1)/  ' 


(9.5) 
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At  this  point,  we  have,  from  Lemma  8.4, 

llM(r)  -  ^(,t(T))||  <  -  l)  <  f . 

Now,  by  construction  we  have  [(ij  <  A/3.  Thus,  we  have  1|<t(T)||  <  A.  Prom  this  point  on, 
a{t)  tends  asymptotically  toward  the  origin. 

Finally,  since  Cl  is  compact,  there  is  a  finite  time  r  >  T(/iO)  for  all  no  €  Cl.  Thus,  we 
can  pick  e  (and  then  a)  to  achieve  the  desired  inequality  for  all  initial  conditions.  □ 

Note  that  if  rj  +  cL  <0,  then  e — and  hence  a — can  be  chosen  constant  for  all  T.  On  the 
other  hand,  if  r?+cl  >  0,  restrictions  on  the  magnitude  of  a  may  only  guarantee  asymptotic 
stability  within  some  finite  distance  from  the  origin. 

It  is  also  important  to  realize  that  the  same  analysis  holds  for  any  other  dynamic  or 
nondynamic  continuous  variable  used  to  approximate  the  max  function,  if  it  is  such  that  it 
can  be  kept  within  e  of  the  max  function  for  arbitrary  e.  (Also  recall  the  note  on  p.  145.) 


§9.3.2  THE  CASE  7  =  0 

For  the  special  case  7  =  0,  the  max  system  continuation  represents  a  switching  between  the 
following  component  linear  systems: 


Ai 


A2 


0  1  0 
—a  —6  1 

0  0  -a^ 

0  1  0 
—a  —6  1 

fa^  goP'  —0? 


Remark  9.11  Both  component  linear  systems  can  be  made  stable  by  choosing  a  large 
enough. 

Proof.  (See  Appendix  9.5.)  D 

Thus  the  component  linear  systems  of  the  max  system  continuation  with  7  =  0  can  be 
chosen  so  both  are  stable.  However,  this  need  not  imply  that  the  whole  system  is  stable. 

NOTE.  A  counterexample  can  be  constructed  after  one  that  appears  in  [139]:  Use  the  asymp¬ 
totically  stable  systems  of  Figures  9-7  and  9-8  (Systems  I  and  II,  respectively),  activating 
System  II  in  quadrants  4  and  2,  System  I  in  quadrants  3  and  1.  Also,  see  Example  8.9. 


The  comparison  arguments  of  the  previous  subsection  do  not  apply  now  since  we  cannot 
find  a  A  like  we  did  there.  Thus,  we  can  only  use  Lemma  8.4  to  get  bounds  on  the 
trajectories  of  the  max  system  continuation.  Note,  however,  that  if  t}  +  cL  <0  then  global 
asymptotic  stability  of  the  max  system  implies  ultimate  boundedness  of  the  max  system 
continuation. 

.  One  may  be  able  to  say  more  about  specific  instances  of  the  max  system  continuation 
(i.e.,  knowledge  of  the  constants).  For  example,  some  cases  may  yield  to  our  robustness 
of  linear  ODEs  lemma  by  comparing  the  case  7  =  0  with  7  =  —  e.  Alternatively,  one 
could  invoke  robustness  results  in  the  literature,  e.g.,  [II,  Theorem  6.1].  These  tools  can’t 
be  invoked  in  the  general  case  because,  roughly,  the  parameter  a  affects  both  77  and  L  in 
conflicting  fashion. 
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Figure  9-3:  Outputs  of  the  tracking  controller:  (a)  normal  acceleration,  (solid),  and 
desired  normal  acceleration,  r  (dashed);  (b)  angle  of  attack,  a  (solid),  and  a’s  limit  (dashed). 


Figure  9-4:  Outputs  of  the  max  controller:  (a)  normal  acceleration,  tIz  (solid),  and  desired 
normal  acceleration,  r  (dashed);  (b)  angle  of  attack,  a  (solid),  and  a’s  limit  (dashed). 
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§9.4  NOTES 

The  work  in  this  chapter  first  appeared  in  [21],  later  summarized  in  [28].  Prof.  Gunter  Stein 
supplied  the  max  system  problem. 


§9.5  APPENDIX 

§9.5.1  MAX  SYSTEM 

Proof,  [of  Fact  9.1]  We  first  need  the  following  theorem  [92,  Theorem  3.10]: 

Theorem  9.12  (Kwakernaak  and  Sivan)  Consider  the  time-invariant  system 

x{t)  =  Ax{t)  +  Bu{t), 
z{t)  =  Dx{t), 

where  z  and  u  have  the  same  dimensions.  Consider  any  asymptotically  stable  time-invariant 
control  law 

u{t)  =  —Fx{t)  +  u'{t). 

Let  H (s)  be  the  open-loop  transfer  matrix 

H{s)  =  D{sI-A)-'^B, 


and  Hc{s)  the  closed-loop  transfer  matrix 

Hcis)  =  D{sl  -  A  +  BF)-^B. 

Then  Hc{0)  is  nonsingular  and  the  controlled  variable  z{t)  can  under  steady-state  conditions 
be  maintained  at  any  constant  value  zq  by  choosing 

u'{t)  =  H:Ho)zo 


if  and  only  if  H{s)  has  a  nonzero  numerator  polynomial  that  has  no  zeroes  at  the  origin. 
For  our  system,  we  have 


CiisI-A)-^B 


0.1(s-9) 
s2  +  2s  +  11’ 


which  has  a  nonzero  numerator  with  no  zeroes  at  the  origin. 


□ 


§9.5.2  MAX  SYSTEM  CONTINUATION 

Proof,  [of  Remark  9.11]  Ai  is  stable  because  it  is  upper  block  triangular  with  stable  blocks. 
One  can  check  that  the  characteristic  polynomial  of  A2  is 

+  (6  +  q2)a2  +  [a  +  a2(6  -  g)]\  +  [a\a  -  /)]  =  +  q'A^  +  /5'A  +  7',  . 

The  Routh  test  (to  verify  stable  roots)  reduces  here  to  [130,  p.  175]: 

1.  a',/3',7'>0, 

2.  /?'  >  '//a'. 
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The  first  of  these  is  verified  by  our  conditions  on  a,b,f,  g,  and  a.  The  second  says  we  need 

a^{a-f) 


a  +  a^{b  —  g)  > 


b  + 


which  reduces  to 


a^{b  —  g)  +  a‘^[b{b  —  g)  +  f]  +  o,b  '  >  0, 


d*  +  o? 


6  + 


/ 


+ 


ab 


>  0. 


(6-5)J  {b-g) 

Since  the  last  term  on  the  left-hand  is  positive  by  our  previous  conditions  (a,  6,  and  6  -  p 
greater  than  zero)  it  is  sufficient  for 

/  1 


-I- 


6  + 


>  0, 


{b-g)\ 

+  b+  .  ^  r  >  0. 

{h-g) 


Again,  since  &  >  0,  it  is  sufficient  for 


o?  +  TT-^  r  >  0, 


(ft -5) 


2  ^  / 
a  > 


[b-g]' 


□ 


Part  III 


Control  of  Hybrid  Systems 


Chapter  10 

Control  of  Hybrid  Systems: 
Theoretical  Results 


In  this  chapter  we  define  an  optimal  control  problem  in  our  unified  hybrid  control  frame¬ 
work  and  derive  some  theoretical  results.  The  necessity  of  our  assumptions— or  ones  like 
them — are  demonstrated  using  examples  throughout  the  sequel.  The  main  results  are  as 
follows:  The  existence  of  optimal  and  near-optimal  controls  for  the  problem  are  established. 
The  “value  function”  associated  with  this  problem  is  expected  to  satisfy  a  set  of  generalized 
quasi-variational  inequalities  {GQVIs),  which  are  formally  derived.  We  conclude  with  a 
brief  list  of  some  of  the  more  striking  open  issues. 

§10.1  THE  CONTROL  PROBLEM 

In  this  section,  we  define  a  control  problem  and  elucidate  all  assumptions  used  in  deriving 
the  results  of  the  sequel. 

§10.1.1  PROBLEM 

Let  a  >  0  be  a  discount  factor.  We  add  to  our  previous  model  the  following  known  maps: 

1.  Running  cost  ki  :  Xi  x  Xi  x  U  M+. 

2.  Autonomous  jump  cost  or  transition  cost  Ca,i :  Ai  xV  -)■  IR+ .  We  may  define 

3.  Controlled  jump  cost  or  impulse  cost  Cc^i  :  Ci  x  D  R+,  satisfying  for  all 
i,j  €  Z+  the  conditions 

Cc{x,y)  >  cq  >  0,  '^x£Ci,yED,  (10-1) 

Cc{x,y)  <  Cc{x,z)  -t-e““'^«(®’^^Cc(2,j/),  'ix£Ci,  zeDr\Cj,  yeD.  (10.2) 


Important  Note.  As  before,  we  have  used  the  shorthand  Cc  :  C  x  2?  ->  1R+ ,  defined 
in  the  obvious  way.  However,  we  have  suppressed  reference  to  the  index  state,  since 
it  is  obvious.  We  do  the  same  below,  with  Ca{x{i),y)  =  Ca^i{x{t),y)  if  x{t)  €  Xi. 
In  such  a  case,  it  is  equivalent  to  think  of  x{t)  as  a  member  of  the  formal  union 
Thus,  to  ease  notation  we  use  this  shorthand  throughout  for  the  maps  G, 
Aa,  Ac,  Ca,  Cc,  etc.  To  alert  the  reader,  such  formal  unions  are  denoted  using  the 
symbol  U. 
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Thus,  autonomous  jumps  are  done  at  a  cost  of  Ca{x{Tj),Vj)  paid  at  time  tj;  controlled 
jumps  at  a  cost  of  Cc{x{Tj),x{rj))  paid  at  time  tj. 

In  addition  to  the  costs  associated  with  the  jumps  as  above,  the  controller  also  incurs  a 
running  cost  of  ki{x{t),x[t],u{t))  per  unit  time  during  the  intervals  [Fj-i.rj),  j  €  Z+.  The 
total  discounted  cost  is  defined  as 

f  e-“^ki{xit),x[t],uit))  dt  +  53e-“‘"*Ca(x(cri),Vi)  +  ^e"“^‘Cc(a:(Ci),a:(Ci)),  (10-3) 

JT  i  i 

where  T  =  I4\(Uih,  Tj)),  {cri}  (respectively  {Ci})  axe  the  successive  pre-jump  times  for  au¬ 
tonomous  (respectively  controlled)  jumps  and  Cj  is  the  post-jump  time  for  the  jth  controlled 
jump.  The  decision  or  control  variables  over  which  Equation  (10.3)  is  to  be  minimized 
are 


•  the  continuous  control  u(-), 

•  the  discrete  controls  {v,},  exercised  at  the  pre-jump  times  of  autonomous  jumps, 

•  the  pre-jump  or  intervention  times  {Ci}  of  controlled  jumps,  and  the  associated 
destinations  {x(Ci)}. 

As  for  the  periods  [Tj,Tj),  we  shall  follow  the  convention  that  the  system  remains  frozen 
during  these  intervals.  Note  that  Equation  (10.1)  rules  out  from  consideration  infinitely 
many  controlled  jumps  in  a  finite  interval  and  Equation  (10.2)  rules  out  the  merging  of 
post-jump  time  of  a  controlled  jump  with  the  pre-jump  time  of  the  next  controlled  jump. 
Our  framework  clearly  includes  conventional  impulse  control  [14]. 

§10.1.2  ASSUMPTIONS 

Throughout  the  sequel,  we  make  use  of  the  following  further  assumptions  on  our  abstract 
model,  which  are  collected  here  for  clarity  and  convenience. 

For  each  i  €  Z+,  the  following  hold:  Xi  is  the  closure  of  a  connected  open  subset  of 
Euclidean  space  d,  €  Z+,  with  Lipschitz  boundary  dXj.  Ai,Ci,  A  C  Xi  are  closed.  In 
addition,  dAi  is  Lipschitz  and  contains  dXi. 

The  maps  G,  Aa,  Ac,  Ca,  and  Cc  are  bounded  uniformly  continuous;  the  ki  are  uniformly 
bounded  and  uniformly  equicontinuous.  The  vector  fields  fi,i£  Z+,  are  bounded  (uniformly 
in  i),  uniformly  Lipschitz  continuous  in  the  first  argument,  uniformly  equicontinuous  with 
respect  to  the  rest.  U,V  are  compact  metric  spaces.  Below,  u(-)  is  a  17- valued  control 
process,  assumed  to  be  measurable. 

All  the  above  axe  fairly  mild  assumptions.  The  following  axe  more  technical  assumptions. 
They  may  be  traded  for  others  as  discussed  in  §10.4.  However,  in  the  sequel  we  construct 
examples  pointing  out  the  necessity  of  such  assumptions  or  ones  like  them. 

Assumption  10.1  d{Ai,Ci)  >  0  and  mf^^j^^diAuDi)  >  0,  d  being  the  appropriate  Eu- 
clidean  distance. 

Assumption  10.2  Each  Di  is  bounded  and  for  each  if  there  exists  an  integer  N{i)  <  oo 
such  that  for  xeCi,ye  Dj,  j  >  N{i),  Cc{x,y)  >  sup^  J{z)'. 
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Assumption  10.3  For  each  i,  dAi  is  an  oriented  -manifold  without  boundary  and  at 
each  point  x  on  dAi,  fi{x,z,u)  is  “transversal”  to  dAi  for  all  choices  of  z,  u.  By  this  we 
require  that  (1)  the  flow  lines  be  transversal  in  the  usual  sense^  and  (2)  the  vector  field  does 
not  vanish  on  dAi. 

Assumption  10.4  Same  as  Assumption  10.3  but  with  Ci  replacing  Aj. 

§10.2  EXISTENCE  OF  OPTIMAL  CONTROLS 

Let  J(x)  denote  the  infimum  of  Equation  (10.3)  over  all  choices  of  tt(-),  {vi},  {Ci})  {a^(Ci)} 
when  x(0)  =  x.  We  have 

Theorem  10.5  A  finite  optimal  cost  exists  for  any  initial  condition. 

Proof.  Let  F,  K,  Q  be  bounds  of  the  fi,  ku  and  Ca,  respectively.  Then,  choosing  to  make 
no  controlled  jumps  and  using  arbitrary  u,  v  we  have  that 

J{x)  <K  e-“‘  dt  +  Y,  Q<K/a-\-QY,  • 

‘'0  i  i 

Let  d  =  i^^i^Z^.d{Ai,Di).  Then  o-j+i  -  <Ti  >  d/F,  so  the  second  term  is  bounded  by 

The  following  corollary  is  immediate  from  the  argument  above: 

Corollary  10.6  There  are  only  finitely  many  autonomous  jumps  in  finite  time. 

To  see  why  an  assumption  like  Assumption  10.1  is  necessary  for  the  above  results,  one 
need  only  consider  the  following  one-dimensional  example: 


Example  10.7  Let  Xi  =  [0, 2],  Ai  =  (0, 2},  and  fi{-,  •,  •)  =  -1  for  each  i  €  Z+.  Also  for 
each  i,  define  Ci  =  %,  Di  =  l/f  and  G{Ai,-)  =  l/{i  -I- 1)^.  Finally,  let  Aa(-,-)  =  0  and 
Ca(‘,  •)  =  1.  Starting  in  Xi  at  x(0)  =  1,  we  see  that 


X 


1 

(iV  +  l)2’ 


Since  the  sum  of  inverse  squares  converges,  we  will  accumulate  an  infinite  number  of  jumps 
and  infinite  cost  by  time  t  = 


Next,  we  show  that  J(x)  is  attained  for  all  x  if  we  extend  the  class  of  admissible  «(•)  to 
“relaxed”  controls.  The  relaxed  control  framework  [155]  is  as  follows:  We  suppose  that 
U  =  ViU'),  defined  as  the  space  of  probability  measures  on  a  compact  space  U'  with  the 
topology  of  weak  convergence  [18].  Also 

fi{x,z,u)  =  J  f-{x,z,u)u{dy),  i  €  Z+, 

ki{x,z,u)  =  j  k[{x,z,u)u{dy),  i  €  Z^., 


^Transversality  implies  that  dAi  is  {di  —  l)-dimensionzJ. 
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for  suitable  {/•},  {k'^}  satisfying  the  appropriate  continuity/Lipschitz  continuity  require¬ 
ments.  The  relaxed  control  framework  and  its  implications  in  control  theory  are  well  known 
and  the  reader  is  referred  to  [155]  for  details. 

Theorem  10.8  An  optimal  trajectory  exists  for  any  initial  condition. 

Proof.  (See  Appendix  10.6.)  ^ 

It  is  easy  to  see  why  Theorem  10.8  may  fail  in  absence  of  Assumption  10.2. 

Example  10.9  Suppose,  for  example,  ki{x,z,u)  =  Oj  and  Ca{x,v)  =  0i  when  x  G  Xi, 
Cc{x,y)  =  7ij  when  x  £  Xu  y  e  Xj,  with  au  0i,  yij  strictly  decreasing  with  i,j.  It  is  easy 
to  conceive  of  a  situation  where  the  optimal  choice  would  be  to  “jump  to  infinity  as  fast  as 
you  can. 

The  theorem  may  also  fail  in  the  absence  of  Assumption  10.3  as  the  following  two-dimensional 
system  shows; 

Example  10.10 

ii(t)  =  1)  xi(0)  =  0, 

X2{t)  =  U,  X2(0)  =  0, 

with  u  €  [0, 1]  and  cost 

[  e“‘ min{lxi(f) -t- X2(t)|,  10^°}  dt, 

Jo 

with  the  provision  that  the  trajectory  jumps  to  [10^®,  10^°]  on  hitting  a  certain  curve  A.  For 
A,  consider  two  possibilities: 

1.  the  line  segment  {xi  =  1,— 1  <  a;2  <  0},  a  C^-manifold  with  boundary; 

2.  the  circle  {(xi,X2)  |  (xi  -  1)^  -h  (x2  +  1)^  =  1},  a  -manifold  without  boundary,  but 
the  vector  field  (1,  u)  with  u  =  0  is  not  transversal  to  it  at  (1, 0). . 

It  is  easy  to  see  that  the  optimal  cost  is  not  attained  in  either  case. 

Also,  it  is  not  enough  that  the  flow  lines  for  each  control  be  transversal  in  the  usual 
sense  3s  the  following  one-dimensional  example  shows. 

Example  10.11  Let  Xi  =  X2  =  IK-f. 

fiix,y,u)  =  -x  +  u,  f2{x,y,u)=0,  u€[-l,0], 

with  running  cost  min{ir,|x|}  and  Gi(0,-)  =  iK,2).  Choosing,  for  example,  K  >  1  one 
sees  that  the  optimal  cost  cannot  be  attained  for  any  1  >  x(0)  >  0. 

Coming  back  to  the  relaxed  control  framework,  say  that  tt(-)  is  a  precise  control  if 
111.)  =  Sg(.\{dy)  for  a  measurable  q  :  [0, oo)  -¥  U'  where  dz  denotes  the  Dirac  measure  at 
z  G  U'.  Let  M  denote  the  set  of  measures  on  [0,T]  x  U'  of  the  form  dt  u{t,  dy)  where  «(•)  is 
a  relaxed  control,  and  Mo  its  subset  corresponding  to  precise  controls.  It  is  known  that  M^o 
is  dense  in  M  with  respect  to  the  topology  of  weak  convergence  [155].  In  conjunction  with 
the  additional  assumption  Assumption  10.4  below,  this  allows  us  to  deduce  the  existence  of 
e-optimal  control  policies  using  precise  «(•),  for  every  e  >  0. 
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Theorem  10.12  Under  Assumptions  10.2-10.4,  for  every  e  >  0  an  e-optimal  control  policy 
exists  wherein  it(-)  is  precise. 

Proof.  (See  Appendix  10.6.)  D 

Remarks.  If  {^(s,  z,y)  |  y  €  U')  are  convex  for  each  z,  a  standard  selection  theorem 
[155]  allows  us  to  replace  by  a  precise  control  which  is  optimal.  Even  otherwise, 

using  Caratheodory’s  theorem  (which  states  that  each  point  in  a  compact  subset  of  R" 
is  expressible  as  a  convex  combination  of  at  most  n  +  1  of  its  extreme  points)  and  the 
aforementioned  selection  theorem,  one  may  suppose  that  for  t  >  0,  the  support  of  u°°{t) 
consists  of  at  most  di  +  1  points  when  x{t)  G  Xi. 


§10.3  THE  VALUE  FUNCTION 

In  the  foregoing,  we  had  set  [0]  =  0  and  thus  a:[0]  =  a:(0)  =  xq.  More  generally,  for 
x(0)  =  xo  €  XiQ,  we  may  consider  x[0]  =  y  for  some  y  €  Xi^,  making  negligible  difference 
in  the  foregoing  analysis.  Let  V{x,y)  denote  the  optimal  cost  corresponding  to  this  initial 
data.  Then  in  dynamic  programming  parlance,  (x,  y)^V (x,  y)  defines  the  “value  function” 
for  our  control  problem. 

In  view  of  Assumption  10.3,  we  can  speak  of  the  right  side  of  dAi  as  the  side  on  which 
/i(-,-,  •)  is  directed  towards  dAi,  i  €  Z+.  A  similar  definition  is  possible  for  the  right  side 
of  dCi  (in  light  of  Assumption  10.4). 

Definition  10.13  Say  that  (xn,y„)  -4  (xoo,yoo)  from  the  right  in  Uj(A’iX Xj)  ifyn  ->•  yoo 
and  either  Xn  Xoo  0  Lli(^^i  U  dC,)  or  x„  Xoo  €  Uii^Ai  U  dQ)  from  the  right  side. 

V  is  said  to  be  continuous  from  the  right  if  (x„,yn)  (xoo,yoo)  from  the  right  implies 
I^(Xn)yn)  ^  I^(Xoo»yoo)- 


Theorem  10.14  V  is  continuous  from  the  right. 

Proof.  (See  Appendix  10.6.)  D 

Corollary  10.15  V  is  continuous  on  lJt(-^t  ^  -^t)\[(^^i  U9Ci)  x  Xi]. 


Again,  Example  10.11  shows  the  necessity  of  the  vector  field’s  not  vanishing  on  dAi. 
Unfortunately,  V  need  not  be  continuous  in  the  data  of  the  hybrid  system: 

Example  10.16  Let  Xi  =  X2  =  R,  /i  =  /2  =  ~  ^2  =  0-  Further,  let 

Ai  =  {p},  Gi(p)  =  (p,  2),  and  c(p,p)  =  0.  Then  we  have 


X  €  X2, 

X  €  Xi,  X  <p, 
X  €  Xi,  X  >  p. 


It  is  clear  that  the  optimal  cost-to-go  is  not  continuous  at  p  in  the  autonomous  jump  set 
data  Ai. 

We  fiball  now  formally  derive  the  generalized  quasi-variational  inequalities  V'(*,  •)  is  ex¬ 
pected  to  satisfy.  Let  C  =  LJi(<^i  -^t)  and  E  C  C  the  set  on  which 


V (x,  y)  =  min  (  Cc(x,  z)-\-e  y (z,  z)  | , 

z^D  ^  * 


(10.4) 
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where  i  €  is  such  that  G  Xi.  For  (x,j/)  G  E,  if  x{t)  —  x  and  x[t]  —  y,  an  optimal 
decision  (not  necessarily  the  only  one)  would  be  to  jump  to  a  z  where  the  minimum  on  the 
right  hand  side  of  Equation  (10.4)  is  obtained.  On  the  other  hand,  for  {x,y)  e  C\E, 

y(x,  y)  <  mm  {cc(a:,  z)  +  2)}  , 

with  i  as  above  and  it  is  not  optimal  to  execute  a  controlled  jump.  For  x  €  Aj,  however, 
an  autonomous  jump  is  mandatory  and  thus 

V(x,  y)  =  min  {ca(x,  v)  +  v),  G(x,  u))}  . 

Suppose  E  is  a  closed  subset  of  Ui(^i  x  ^i)-  Let  If  =  F  U  (Ui(Ai  x  Xi)),  with  M  = 
(1J.(X,  X  Xi))\ff.  Let  (x,y)  6  M°,  with  x,y  €  Xi^  (say).  Let  O  be  a  bounded  open 
neighborhood  of  (x,  y)  in  M°  with  a  smooth  boundary  dO  and  u  =  inf{t  >  0  |  (x(f),  y)  ^O}, 
where  x(-)  satisfies 

ic(<)  = x(0)  =  x,t  €  [0,1^].  (10.5) 

Note  that  y  is  a  fixed  parameter  here.  By  standard  dynamic  programming  arguments, 
V{x,y),  X  G  O,  y  as  above,  is  also  the  value  function  for  the  “classical  control  problem  of 
controlling  Equation  (10.5)  on  [0,  with  cost 

/  e““‘A:jo(x(t),t/,u(f))  dt  +  e~°'''h{x{u),y), 

Jo 

where  fi(-,-)  s  F(-,-)  on  dO.  It  follows  that  F(x,y),  (x,y)  €  O  is  the  viscosity  solution 
of  the  Hamilton- Jacobi  equation  for  this  problem  [52],  i.e.,  it  must  satisfy  (in  the  sense  of 
viscosity  solutions)  the  p.d.e. 

mm{{V^V{x,y),fi^{x,y,u)) -aV{x,y)  +  ki^{x,y,u)}  =  0  (10.6) 

U 

in  O  and  hence  on  M".  (Here  Vx  denotes  the  gradient  in  the  x  variable.)  Elsewhere, 
standard  dynamic  programming  heuristics  suggest  that  Equation  (10.6)  holds  with  ‘=’ 
replaced  by  ‘<’. 

Based  on  the  foregoing  discussion,  we  propose  the  following  system  of  generalized 
quasi- variational  inequalities  for  V^(",-);  For  (x,y)  E  Xi  x  Xi, 

V{x,y)  <  mm{cc(x,2)  +  e““^‘=^®’^V(2,2)}  on  C  (10-7) 

V{x,y)  <  nun{ca(a:,t;)  +  e-“^“(*-"V(G(x,t;),G(x,t;))}  on  Ui(AiXXi)  (10.8) 

min{{V^V{x,y),fi{x,y,u))-aV{x,y)  +  ki{x,y,u)}<0  (10.9) 

U 

^V(x,y)  -minxgD  |cc(x,2)  -I- (10.10) 

•  (minu  {(VxF(x,  y),  /i(x,  y,  u))  -  aV (x,  y)  +  ki{x,  y,  u)})  =  0  on  C 

(Equation  (10.10)  states  that  at  least  one  of  Equation  (10.7),  Equation  (10.9)  must  be 
an  equality  on  C.)  (Equations  (10.7)-(10.10)  generalize  the  traditional  quasi-variational 
inequalities  encountered  in  impulse  control  [14].  We  do  not  address  the  issue  of  well- 
posedness  of  Equations  (10.7)-(10.10). 
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The  following  “verification  theorem,”  however,  can  be  proved  by  routine  arguments. 

Theorem  10.17  Suppose  Equations  (10.7) -(10.10)  has  a  “classical”  solution  V  which  is 
continuously  differentiable  “from  the  right”  in  the  first  argument  and  continuous  in  the 
second.  Suppose  x{-)  is  an  admissible  trajectory  of  our  control  system  with  initial  data 
{xQ,yo)  andu{-),  {ui},  {cr*},  {Ci}»  {n],  {Fi}  the  associated  controls  and  jump  times,  such 
that  the  following  hold: 

1.  For  a.e.  t  G  T,  i  such  that  x{t)  G  Xi, 

(VxF(x(t),a;[f]),/i(x(<),x[t],u(t)))  +fci(x(<),x[t],u(t))  = 

min  {(VxV"(x(t),  x[t]),  /i(x(<),  x[t],  u))  +  fei(x(<),  x[t], «)}  . 

U 


2.  For  all  i, 

V{x{ai),x[<7i])  -  Ca{x{ai),Vi)  +exp{-aAa(x(cri),Vi)}y(G(x((Ti),Vi),G’(x(cri),r;i)). 

3.  For  all  i, 

F(x(Ci),x[Ci])  =  Cc(x(Ci),x(Ci))  +exp{-oAc(x,x(C^))}^(a;(Ct')>a;(C^))• 

^/^en  x(-)  is  an  optimal  trajectory. 

§10.4  DISCUSSION 

The  foregoing  presents  some  initial  steps  towards  developing  a  unified  “state  space”  paradigm 
for  hybrid  control.  Several  open  issues  suggest  themselves.  We  conclude  with  a  brief  list  of 
some  of  the  more  striking  ones. 

1.  A  daunting  problem  is  to  characterize  the  value  function  as  the  imique  viscosity  solu¬ 
tion  of  the  generalized  quasi- variational  inequalities  Equations  (10.7)-(10.10). 

2.  Many  of  our  assumptions  can  possibly  be  relaxed  at  the  expense  of  additional  techni¬ 
calities  or  traded  off  for  alternative  sets  of  assumptions  that  have  the  same  effect.  For 
example,  the  condition  d{Ci,Ai)  >  0  could  be  dropped  by  having  Cc  penalize  highly 
the  controlled  jumps  that  take  place  too  close  to  Ai.  (In  this  case.  Assumption  10.4 
has  to  be  appropriately  reformulated.) 

3.  Example  10.10  show  that  Assumption  10.3  cannot  be  dropped.  In  the  autonomous 
case,  however,  the  set  of  initial  conditions  that  hit  a  C®®  manifold  are  of  measure  zero 
[135].  Thus,  one  might  hope  that  an  optimal  control  would  exist  for  almost  all  initial 
conditions  in  the  absence  of  Assumption  10.3.  The  system  of  Example  10.11  showed 
this  to  be  false.  Likewise,  in  the  systems  of  Example  10.10  we  have,  respectively,  no 
optimal  control  for  the  sets 

{(xi,X2)  1  X2  <  0,Xi  <  1,X2  +  1  >  xi}, 

{(xi,X2)  I  X2  <  0,xi  <  o,X2  +  a  >  Xi}  U  ^[0, 1]  X  [—a/2,0]  —  B([l,  —1]^,  1))  , 
where  a  =  2  —  \/2  and  B{x,  r)  denotes  the  ball  of  radius  r  about  the  point  x. 


162  Control  of  Hybrid  Systems:  Theoretical  Results 


It  remains  open  how  to  relax  the  conditions  Assumptions  10.3  and  10.4.  This  might 
be  accomplished  through  additional  continuity  assumptions  on  G,  Ao,  and  c^. 

4.  An  important  issue  here  is  to  develop  good  computational  schemes  to  compute  near- 
optimal  controls,  which  is  currently  a  topic  of  further  research.  See  [51,  §11-1]  for 
some  related  work. 

This  is  a  daunting  problem  in  general  as  the  results  of  [25,  §7]  show  that  the  hybrid 
systems  models  discussed  in  §3  can  simulate  arbitrary  Thring  machines  (TMs),  with 
state  dimension  as  small  as  three.  It  is  not  hard  to  conceive  of  (low-dimensional) 
control  problems  where  the  cost  is  less  than  1  if  the  corresponding  TM  does  not  halt, 
but  is  greater  than  3  if  it  does.  Allowing  the  possibility  of  a  controlled  jump  at  the 
initial  condition  that  would  result  in  a  cost  of  2,  one  sees  that  finding  the  optimal 
control  is  equivalent  to  solving  the  halting  problem. 

5.  Another  possible  extension  is  in  the  direction  of  replacing  Xi^  by  smooth  manifolds 
with  boundary  embedded  in  a  Euclidean  space.  See  [35]  for  some  related  work. 

6.  In  light  of  Definition  10.13,  all  the  proofs  seem  to  hold  if  Assumption  10.1  is  relaxed 
to  only  consider  distances  “from  the  right,”  that  is  if  mfid+{Ai,Di)  >  0,  with 

d+{Ai,Di)  =  inf  Ejix,u{-))  €  At, 

t>0,u(),x6Di 

where  Ej{x,u{-))  denotes  the  solutions  under  /,  with  initial  condition  x  and  control 
ii(-)  in  Here,  time  can  be  used  as  a  “distance”  in  light  of  the  imiform  bound 

on  the  /t;  we  consider  t  >  0  by  adding  that  caveat  that  if  we  jump  directly  onto  Ai, 
we  do  not  make  another  jump  until  we  hit  it  again.  Presumably  one  must  also  make 
some  transversality  or  continuity  assumptions  for  well-posedness.  This  would  allow 
the  results  to  extend  to  many  more  phenomena,  including  those  examples  in  [35]. 

§10.5  NOTES 

The  results  of  this  chapter  grew  out  of  [105].  They  are  joint  work  with  Profs.  Vivek  S. 
Borkar  and  Sanjoy  K.  Mitter  completed  in  December  1993.  They  appeared  in  [32]. 

Hybrid  control  is  a  rapidly  expanding  field  and  we  make  no  explicit  reference  to  general 
papers  here  (see  §3  and  the  references).  Viable  control,  considered  by  [57,  86],  was  not 
discussed  here.  However,  optimal  control  of  hybrid  systems  has  been  considered  in  [95]  (for 
the  discrete-time  case)  and  the  pioneering  work  of  [114].  Kohn  is  the  first  we  know  of  to  use 
relaxed  controls  and  their  e-optimal  approximations  in  the  hybrid  systems  setting  (see  [114, 
Appendix  I]  and  the  references  therein).  The  algorithmic  importance  of  these  was  further 

described  in  [61]. 

We  have  already  mentioned  the  link  to  impulse  control.  Even  closer  are  the  results 
of  [154],  discovered  after  this  work  was  completed.  That  paper  considers  switching  and 
“impulse  obstacle”  operators  akin  to  those  in  Equations  (10.7)  and  (10.8)  for  autonomous 
and  (controlled)  impulsive  jumps,  respectively.  Yong  restricts  the  switching  and  impulse 
operators  to  be  uniform  in  the  whole  space,  which  is  unrealistic  in  hybrid  systems.  However, 
he  derives  viscosity  solutions  of  his  corresponding  Hamilton- Jacobi-Bellman  system.  His 
work  may  be  useful  in  deriving  viscosity  solutions  to  our  GQVIs. 

Also  after  this  work  was  completed,  we  became  aware  of  the  model  and  work  of  [152, 
§3.3].  In  that  paper,  Witsenhausen  considers  an  optimal,  terminal  constraint  problem  on 
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his  hybrid  systems  model.  Recall  his  model  contains  no  autonomous  impulses,  no  controlled 
switching,  and  no  controlled  jumps. 
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Proof,  [of  Theorem  10.8]  Fbc  x(0)  =  xq  €  Xi^,  io  €  Z+.  Consider  a  sequence 

!»"(■),  W},  {<}.  {C”),  {Tr},  {r?}),  ns  Z+, 

associated  with  our  control  system,  with  the  obvious  interpretation,  such  that  z”(0)  =  xq 
for  all  n  and  the  corresponding  costs  decrease  to  J{xo).  Let  y”(’)  denote  the  solution  of 

nt)  =  «”(*)),  y”(0)  =  xo,n  €  Z+.  (10.11) 

Then  x"(-),  y"(-)  agree  on  [0,Tf).  Since  {/,}  are  bounded,  {y”(-)}  are  equicontinuous 
bounded  in  C(IR+;R‘^*o ),  hence  relatively  sequentially  compact  by  the  Arzela-Ascoli  theo¬ 
rem.  The  finite  nonnegative  measures  rf'{dt,dy)  =  dt  u'^{t,dy)  on  [0,T]  x  U'  are  relatively 
sequentially  compact  in  the  topology  of  weak  convergence  by  Prohorov’s  theorem  [18].  {t”} 
are  trivially  relatively  compact  in  [0,  oo].  Thus  dropping  to  a  subsequence  if  necessary,  we 
may  suppose  that  y"(-)  -)•  y~(-),  rr{dt,dy)  r)°°{dt,dy),  ^  rf®  in  the  respective 

spaces.  Clearly  t}°°  disintegrates  as  rf^{dt,dy)  =  dt  u°°{t,dy).  Rewrite  Equation  (10.11)  as 


y"(<)  =  xo-f 

fioiy"'is),xo,u^{s))  -  fio{y°°{s),xo,u^{s))ds'^  +J^  fio{y°°{s),xo,u^is))ds 

for  <  >  0.  By  the  uniform  Lipschitz  continuity  of  fi^,  the  term  in  parentheses  tends  to 
zero  as  n  -¥  oo.  Since  y"  t?"®,  the  last  term,  in  view  of  the  relaxed  control  framework, 
converges  to 

f  fio{y°°is),xo,y'°°is))ds 
Jo 

for  t  G  [0,T].  Since  T  was  arbitrary  a  standard  argument  allows  us  to  extend  this  claim  to 
t  €  [0,  oo).  (We  use  [18,  Theorem  2.1(v),  p.  12]  and  the  fact  that  v°°{{t}  x  U')  =  0.)  Hence 
y°°(-),  u~(-)  satisfy  Equation  (10.11)  with  n  =  oo.  Since  d{Cio,Ai^)  >  0,  either  r/*  =  < 
for  sufficiently  large  n,  or  r”  =  for  sufficiently  large  n.  Suppose  the  first  possibility 
holds.  Then  y°°(Tf°)  =  limx"(T{‘)  €  Ai^.  Let  vf°  in  V  along  a  subsequence. 

Then  Ca(x'‘(Tr),v5*)  Ca(y~(ri~), vf),  Aa(x"(rr),  v?)  Aa(y~(Tp),vr),  P?  ^  Pf  = 

Tf°-I-Aa(y~(rf°),  vf°).  Set  x~(-)  =  y~(-)  on  [0,Tf°]  and  x~(Pf )  =  G(x“(Tf°),t;f ).  Then 


n  e~‘'^kio{x'^{t),xo,u'^{t))dt-^  e  “*A:io(x°°(f),xo,u~(f))  df.  (10.12) 

Jo  Jo 

If  the  second  possibility  holds  instead,  one  similarly  has  yi(Tf°)  €  Qq.  Then  As¬ 
sumption  10.2  ensures  that  {x"(P5*)}  is  a  bounded  sequence  in  D  and  hence  converges 
along  a  subsequence  to  some  y'  €  D.  Then,  on  dropping  to  a  further  subsequence  if 
necessary,  Cc(x”(tI*),x"(P?)  ->  Ce(y~(T{*),y'),  Ac(x"(Ti*),x"(P?))  Ac(y'*(Tf),y').  Set 

x~(.)  =  y°®(-)  on  [0,  Tf°],  Pf  =  rf  -h  Ac(y°°(rf°),y'))  =  UmP?  and  x~(Pf )  =  y'.  Again 
Equation  (10.12)  holds.  Note  that  in  both  cases,  x~(-)  defined  on  [0,Tf°]  is  an  admissible 
segment  of  a  controlled  trajectory  for  our  system.  The  only  way  it  would  fail  to  be  so  is  if 
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it  hit  Aio  in  [0,Tf°).  If  so,  x"(-)  would  have  to  hit  Ai^  in  [0,rf)  for  sufficiently  large  n  by 
virtue  of  Assumption  10.3,  a  contradiction. 

Now  repeat  this  argument  for  {x’^(rf  +  •)}  in  place  of  {x”(-)}.  The  only  difference  is 
a  varying  but  convergent  initial  condition  instead  of  a  fixed  one,  which  causes  only  minor 
alterations  in  the  proof.  Iterating,  one  obtains  an  admissible  trajectory  x°°(-)  with  cost 
J(xo).  ° 

Proof,  [of  Theorem  10.12]  Recall  the  setup  of  Theorem  10.8.  Consider  the  time  inter¬ 
val  [0,rf°].  Let  «"(•),  n  e  Z+,  be  precise  controls  such  that  dt  lP{t,dy)  -4  T]°°{dt,dy)  = 
dt  u°°{t,dy)  in  the  topology  of  weak  convergence.  Let  y^{-),  n  G  Z+,  denote  the  corre¬ 
sponding  solutions  to  Equation  (10.11).  Now  rf°  equals  either  a”  or  Suppose  the 
former  holds.  As  in  the  proof  of  Theorem  10.8,  we  have  y^  -4  in  C([0,  oo),Xi(,). 

Using  Assumption  10.3  as  in  the  proof  of  Theorem  10.8,  one  verifies  that 

=  inf{f  >  0  I  Tit)  G  AiJ  -4 
Thus  for  any  5  >  0,  we  can  take  n  large  enough  such  that 

sup{iirw-y~wii  <  <5, 

K-bA„(r(^?),t^r)-rri  <  <5- 

Set  =  r(-)  on  [0,5^]  and  -H  Aa(r*(5^),0)  =  (corresponding 

to  control  action  vj®).  The  latter  may  be  taken  to  lie  in  the  open  <5- neighborhood  of  x°®(rf° ) 
by  further  increasing  n  if  necessary.  In  case  rf®  =  Ci° >  one  uses  Assumption  10.4  instead 
to  conclude  that  Tit'n)  ^  ^io  for  some  in  the  5-neighborhood  of  rf®  for  n  sufficiently 
large.  Set  ^  x^(-)  =  Ti')  on  [0,^")-  By  further  increasing  n  if  necessary,  we  may 

also  ensure  that 

{x”(t)  |tG[0,cf)}n_Aio  =  0, 
sup{||y"W-a;~(f)ll  |0<f<^ACr}  < 

Ae(x”(^),x°°(rr))-rri  <  6. 

Set 

x"(^  +  Ae(x"(^),x°®(rr)))  =  x°®(rr). 

It  is  clear  how  to  repeat  the  above  procedure  on  each  interval  between  successive  jump  times 
to  construct  an  admissible  trajectory  x"(-)  with  cost  within  e  of  J(xo)  for  a  given  e  >  0.  □ 

Proof,  [of  Theorem  10.14]  Let  (xn,j/n)  {xoo^Voo)  from  the  right  in  x  Xi)  and 
let  c^i-),  n  G  Z+  U  {oo},  denote  optimal  trajectories  for  initial  data  (xn,y„)  respectively. 
By  dropping  to  a  subsequence  of  n  G  Z+  if  necessary,  obtain  as  in  Theorem  10.8  a  limiting 
admissible  trajectory  x'(-)  for  initial  data  (xoo,yoo)  with  cost  (say)  a  such  that  V{xn,yn) 
a  >  V{xoc,  yoo).  Suppose  a  >  U(xoo,  yoo)  +  3e  for  some  e  >  0.  Starting  from  x°°(-),  ar^e  as 
in  Theorem  10.12  to  construct  a  trajectory  x”(-)  with  initial  data  (x„,y„)  for  n  sufficiently 
large,  so  that  the  corresponding  cost  does  not  exceed  V{xoo,yoo)  +  e.  At  the  same  time, 
y(a;„,  y„)  >  a  -  e  >  U(xoo,  yoo)  +  2e  for  n  sufficiently  large,  which  contradicts  the  fact  that 
V{xn,yn)  is  the  optimal  cost  for  initial  data  (x„,yn).  The  claim  follows.  D 


Chapter  11 

Hybrid  Control  Algorithms  and 
Examples 


In  this  chapter  we  outline  four  approaches  to  solving  the  generalized  quasi-variational  in¬ 
equalities  associated  with  optimal  hybrid  control  problems.  Then  we  solve  some  illustrative 
problems  in  our  framework. 

§11.1  ALGORITHMS  FOR  OPTIMAL  HYBRID  CONTROL 

In  §5,  we  proposed  a  very  general  framework  for  hybrid  control  problems  that  was  shown  to 
encompass  all  hybrid  phenomena  considered  in  this  thesis  and  all  hybrid  models  reviewed 
from  the  literature.  A  specific  control  problem  was  studied  in  this  framework,  leading  to  an 
existence  result  for  optimal  controls.  The  value  function  associated  with  this  problem  was 
seen  to  satisfy  a  set  of  generalized  quasi-variational  inequalities  (GQVIs).  In  this  section, 
we  give  explicit  algorithms  for  computing  the  solutions  to  such  optimal  control  problems. 

There  are  two  foundations  to  these  algorithms.  First,  our  unified  view — treating  con¬ 
tinuous  and  discrete  controls  in  a  conceptually  similar  manner — led  us  to  a  generalized 
Bellman  equation  that  may  be  solved  via  adaptations  of  standard  methods:  value  and 
policy  iteration. 

Second,  as  previously  noted  [32,  §10],  the  key  to  efficient  algorithms  for  solving  optimal 
control  problems  for  hybrid  systems  lies  in  first  establishing  their  strong  connection  to  the 
models  of  impulse  control  and  piecewise-deterministic  processes  (PDFs)  [14,  51,  53,  156, 
§2.2.4,  §2.2.5]  Then,  we  modify  the  algorithms  that  have  been  found  useful  in  solving  those 
problems  [50,  51].  The  result  is  an  impulse  control-like  algorithm  and  a  linear  programming 
solution  for  computing  optimal  controls. 

The  object  of  this  section  is  to  make  the  above  observations  more  precise.  Throughout 
we  follow  an  amalgamation  of  the  notation  of  [51,  §10].  To  ease  notation  we  do  not  discuss 
the  general  case  considered  in  §10,  i.e.,  we  assume  V  is  only  a  function  of  the  state  (and 
not  also  the  last  jump  point). 

§11.1.1  BOUNDARY-VALUE  ALGORITHMS 

We  now  discuss  explicit  solutions  of  the  GQVIs.  First,  note  that  instead  of  thinking  of  V 
defined  on  the  generalized  state-space  in  the  previous  section,  V  :  ->■  K+,  we  may 

revert  to  considering  it  component-wise:  1^  :  Xj  ->  Rf . 

The  Boundary- Value  algorithms  outlined  in  this  section  begin  with  a  guess  for  each  Vi. 
The  algorithm  is  summarized  as  follows: 
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1.  Given:  a  guess  for  each  Vi. 

2.  For  each  i,  compute  the  constrained  boundary  values  arising  from  the  GQVIs: 

(a)  For  X  e  Ai  impose  the  constraint 

Fi(x)  =  mm  {ca(x,u)  + 

(b)  For  X  eCi  impose  the  constraint 

Fi(x)  =  min|v'i(x),  mm{cc(x,2:)  +  , 

3.  Solve  the  GQVI  Equation  (10.9)  to  update  each  Vi  separately,  imposing  the  constraints 
of  Step  2.  This  is  an  optimal  control  problem  as  we  are  used  to  encountering. 

4.  If  convergence,  exit;  else  go  to  Step  2. 

Note  that  the  boundary  value  problems  in  Step  3  can  be  solved  in  a  variety  of  ways  thern- 
selves,  including  value  iteration,  policy  iteration,  or  (as  in  the  case  of  linear  systems)  explicit 
methods  [121].  As  is  usual  in  boundary-value  problems,  convergence  in  Step  4  is  determined 
by  comparing  computed  and  constrained  values.  We  have  made  no  attempt  to  theoretically 
study  the  above  algorithm  for  convergence,  but  we  have  used  it  to  solve  the  hysteresis 
example  discussed  below. 

§11.1.2  GENERALIZED  BELLMAN  EQUATIONS 

Another  solution  method  arises  from  our  unified  viewpoint.  The  algorithmic  basis  is  the 
following  Bellman  Equation: 

V*{x)  =  min{5(x,a)  -t-  y*(x'(x,  a))}  ,  (11-1) 

pen 

where  11  is  a  generalized  set  of  actions.  The  three  classes  of  actions  available  in  our  hybrid 
systems  framework  at  each  x  are 

•  Continuous  Controls,  u  &U. 

•  Autonomous  Jumps,  possibly  modulated  by  discrete  controls  v  eV  (if  x  €  A). 

•  Controlled  Jumps,  choosing  source  and  destination  (if  x  G  C). 

To  solve  on  a  computer,  we  first  (discretize  the  continuous  state  and  controls  and) 
compute  for  x  e  Xi  the  minimum  of  the  following  three  quantities: 

fi{x{t),u)  dt 

min  {ca(x,u)  +  e““^“^*’"^V(G(x,t;)),  | 
vev  t  ■’ 

min(cc(x,z)-be-“^'(*-"V(z)}, 
z€D  ^  ^ 


ki{x{t),u)  dt  +  V 
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with  several  caveats.  First,  we  make  sure  that  we  do  not  hit  A  in  our  computation  of  the 
first  quantity.  (Briefiy,  if  we  do,  we  replace  S  by  —  t.)  Second,  the  last  two  quantities 
are  not  taken  into  account  if  x  ^  A  or  x  ^  C,  respectively. 

The  resulting  system  may  be  solved  via  value  or  policy  iteration.  So  far,  we  have 
concentrated  on  value  iteration  (solved  via  relaxation)  and  call  the  resulting  algorithm 
“Generalized  Value  Iteration.” 

More  specifically,  the  algorithm  is  as  follows: 

1.  Discretize  the  state  space  into  X  and  set  of  continuous  controls  into  U. 

2.  For  each  x  €  A’,  guess  a  value  for  V(x),  e.g.,  zero. 

3.  For  each  x  €  A,  compute  the  minimum  of  the  three  quantities  described  above, 
subject  to  the  caveats.  The  first  minimum  is  taken  with  respect  to  u  €  U  with  the 
cost-to-go  a  weighted  sum  of  the  costs-to-go  of  nearest  neighbors.  Set  V{x)  equal  to 
that  minimum.  Repeat  until  convergence. 

We  do  not  go  into  theoretical  results  here,  but  only  note  that  the  theoretical  firamework 
of  [45]  is  very  general,  allowing,  e.g.,  piecewise  Lipschitz  continuous  dynamics. 

§11.1.3  ALGORITHMS  INSPIRED  FROM  IMPULSE  CONTROL 

We  now  make  more  explicit  the  strong  connection  between  hybrid  systems  and  the  models 
of  impulse  control  and  piecewise-deterministic  processes  (PDFs)  §§2.2.4-2.2.5.  More  specif¬ 
ically,  the  controlled  jumps  sets  of  our  model  axe  precisely  those  sets  in  the  impulse  control 
framework  where  r(x)  7^  0.  The  associated  controlled  jump  costs  are  likewise  completely 
analogous  to  those  of  impulse  control.  The  connection  with  autonomous  jump  sets  is  only 
slightly  less  direct:  they  can  viewed  as  “singular  limits”  of  the  jump  measures  A  and  A  of  the 
the  theory  of  PDFs.  We  have  added  the  possibility  of  autonomous  jump  costs  associated 
with  such  jumps.  The  jump  destination  sets  are  merely  the  union  of  the  points  that  one 
can  jump  to  using  either  controlled  or  autonomous  jumps.  We  have  also  added  autonomous 
and  controlled  jump  transition  delay  maps,  but  these  present  no  conceptual  challenges,  and 
are  omitted  from  the  following  discussion. 

One  conceptual  differences  is  the  inclusion  of  continuous  controls  in  our  model.  The 
other  conceptual  difference  is  that  instead  of  having  a  single,  continuous  state  space,  e.g, 
X  =  K.”,  we  have  an  indexed  set  of  continuous  state  spaces,  X  =  {Afi}^o  where  each  Xi  is 
a  subset  of  some  Euclidean  space  ,  di  €  Z+. 

The  splitting  into  discrete  and  continuous  states  is  not  a  technical  barrier.  Indeed, 
the  algorithms  that  arise  from  this  structure  are  a  natural  consequence  of  the  underlying 
“automata”  interpretations  (for  example,  our  approximation  techniques  may  be  viewed  as 
“unrolling”  these  automata  into  trees). 

Now,  we  show  how  the  algorithms  in  [51]  may  be  modified  to  form  the  basis  of  control 
algorithms  for  hybrid  systems.  Generalizing  from  [51]  to  the  case  of  our  hybrid  systems 
model,  define  the  following  operators: 

raiAt 

J{VuV2){t,x)  =  /  e"“®A:(x(s))  ds 

-^e-“‘Vi(x(f))l{t<^,} 

min{ca(x(cri))  -t-  V2(G(x(<ti)))}  •  l{<ri<t} 
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ICV2{x) 


C{VuV2){x) 

MV2(x) 

AV2ix) 

h{x) 


=  f  \~°‘^k{x{s))  ds  +  e  min{ca(x((Ti))  +  T^2(G'(x(cti)))} 

Jo 

=  J(Vi,V'2)(cx),x) 

=  (  inf  J{VuV2){t,x)]  AlCV2ix) 


=:  inf  {c(x,y)  +  V'2(y)} 

yer(x) 


=  inf 

r€Mc 


e  °‘^k{xs)  ds  +  e  “‘A1F2(xt)| 

/*oo 

=  /  e~°‘^k{xs)  ds 

Jo 


Above,  the  set  Moo  is  the  set  of  all  times  we  are  in  the  controlled  jump  set  C.  Finally, 
define  L{V)  =  C{MV,  F). 

The  following  results  may  then  be  derived  analogously  to  the  results  in  [51]. 
Proposition  11.1  h  is  the  smallest  solution  of 


{  V  =  KV 

I  y  >  0 


and  h(j  =  0,  hn-\-i  —  IChn  implies  hn  ^  h  as  n  ^  oo. 

Proposition  11.2  V*  is  the  biggest  solution  ofV  =  AV  and  V*  =  lim„_voo 

Proposition  11.3  V*  is  the  biggest  solution  of 

/  V  =  L{V) 

\  V  <  h 


and  Vo  =  h,  Vn+i  =  L{Vn)  implies  VniV*  as  n -i-  oo. 

Proposition  11.4  Assume  Vo  >  h,  In+i  =  L{Vn).  Then  limn->oo  Ki(x)  =  V  (x),  for  all 

X. 

We  can  thus  use  the  L  iterations  above  to  compute  the  value  function  solving  our 
GQVIs.  Continuous  controls  u  are  taken  care  of  by  discretizing  the  control  space  into  a 
finite  number  of  values,  treating  the  current  value  of  u  as  part  of  an  augmented  state,  and 
assigning  a  switching  cost  for  transition  between  different  control  values.  So,  we  perform  L 
iterations  on  this  problem.  If  performance  of  the  resulting  is  satisfactory,  we  conclude.  Else, 
we  may  decrease  the  cost  of  switching  among  continuous  controls  or  increase  the  number 
of  quantized  control  values  considered.  We  refer  to  the  above  algorithm  as  the  L-Iteration 
algorithm. 

Results  on  convergence  of  discretization  of  algorithms  should  follow  similarly  to  those 
for  impulse  control  [51]. 

§11.1.4  LINEAR  PROGRAMMING  SOLUTIONS 

Linear  programming  has  been  used  to  solve  for  the  value  function  associated  with  optimal 
control  of  Markov  decision  processes  [124].  It  has  also  been  shown  to  be  valuable  in  impulse 
control  of  PDFs  [50].  Here  we  use  it  to  solve  hybrid  control  problems. 
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Again  assume  we  have  discretized  the  state  space  into  X  and  the  set  of  continuous 
controls  into  U  (yielding  |ni  finite).  Consider  the  right-hand  side  of  Equation  (11.1)  as  an 
operator,  T,  on  functions  V.  Arguing  as  in  [124,  Lemma  6.21,  p.  151],  monotonicity  of  T 
and  V*  =  TV*,  imply  that  V*  may  be  obtained  by  solving 

Maximize  V  subject  to  TV  >  V . 

However,  since  maximizing  V{x)  for  each  x  €.  X  also  maximizes  12^(2:))  problem 
reduces  to 

Maximize  ^  V  (x)  subject  to 

minaen  {^(a:,  a)  +  V'(x'(x,  a))}  >  V"(x),  for  all  x  €  A. 

Alternatively,  we  can  write  this  as 
Maximize  ^  V  (x)  subject  to 

{g(x,a)  +  V{x'{x,a))}  >  V^(x),  for  all  a  €  11  and  x  €  A. 

This  is  just  a  linear  program  and  may  be  solved  by  standard  techniques.  The  problem 
is  that  these  linear  programs  may  be  large.  If  \X\  =  N,  and  in|  =  M,  then  we  have  N 
variables  and  MN  constraints.  The  advantage  is  that  the  matrix  of  constraints  is  sparse. 
Briefly,  this  is  since  the  row  corresponding  to  (x,  a)  only  has  non-zero  weights  for  the  nearest 
neighbors  of  the  state  x'{x,a). 

§11.2  CONTROLLING  EXAMPLE  SYSTEMS 

We  discuss  below  the  solution  of  several  simple,  but  illustrative  control  problems.  We  begin 
with  a  quick  warmup.  Then  we  discuss  continuous  control  of  a  hysteresis  system  that 
exhibits  autonomous  switching.  Then  we  discuss  a  satellite  control  problem.  The  on-off 
nature  of  the  satellite’s  reaction  jets  creates  a  system  involving  controlled  switching.  We 
end  with  a  transmission  problem.  The  goal  is  to  find  the  strategy  of  continuous  accelerator 
input  and  discrete  gear-shift  position  to  achieve  maximum  acceleration. 

WARMUP 

First,  going  back  to  Example  10.11  we  have 

Example  11.5  Consider  Example  10.11  except  with  the  controls  restricted  in  [-1, 
e  <  1.  Then  the  flows  are  transversal  and  do  not  vanish  on  Ai  —  {0}  for  any  u. 
case,  the  optimal  control  exists.  For  example,  if  K  >  1/e,  one  can  show  that  u(-) 
optimal. 

§11.2.1  HYSTERESIS  EXAMPLE 

We  now  consider  a  hybrid  control  example  which  combines  continuous  control  with  the 
phenomenon  of  autonomous  switching. 

Example  11.6  Consider  a  control  system  with  hysteresis: 

X  =  f{x,u)  =  H(x)  -hu, 

where  the  multi-valued  function  H  is  shown  in  Figure  11-1. 


-e],  0  < 
In  this 
=  — e  is 
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H 


Figure  11-1:  Hysteresis  Function. 

Note  that  this  system  is  not  just  a  differential  equation  whose  right-hand  side  is  piecewise 
continuous.  There  is  “memory”  in  the  system,  which  affects  the  value  of  the  vector  field.  In¬ 
deed,  such  a  system  naturally  has  a  finite  automaton  associated  with  the  hysteresis  function 
H,  as  pictured  in  Figure  11-2. 
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Figure  11-2:  Finite  Automaton  Associated  with  Hysteresis  Function. 

As  a  control  problem  we  consider  minimizing 

J  =  r  =  r  k{x,u)e-^'dt  (11.2) 

Jo  2  Jo 

Let  s  =  H{x).  We  first  solve  for  V(x,s),  and  then  for  u.  By  symmetry,  we  expect 
F(-A,  1)  =  V{A,  -1).  From  the  GQVIs,  we  expect  V  to  satisfy 

minu{-aV{x,$)  +Vxix,s)  -  f{x,u)  +  k{x,u)}  =  0, 

F(A,1)  =c-l-l^(A,-l),  (11-3) 

y(-A,-l)  =  c  +  F(-A,l), 

where  s  takes  on  the  values  ±1  and  c  represents  the  cost  associated  with  the  autonomous- 
switchings. 

We  have  solved  these  equations  numerically  (via  the  boundary-value  algorithm)  for  the 
case  c  =  0,  o  =  1,  A  =  0.1.  The  boundary-value  algorithm  outlined  in  §11.1  was  used.  The 
resulting  control  u,  plotted  against  x  (for  s  =  1)  and  q,  is  shown  in  Figure  11-3.  As  the 
state  is  increasingly  penalized,  the  control  action  increases  in  such  a  way  to  invert  the 
hysteresis  function  H. 

To  get  an  idea  of  the  dynamics  we  plot  the  state  and  control  over  time  for  several  values 
of  q  in  Figure  11-4  and  Figure  11-5,  respectively.  As  the  penalty  on  the  state  increases, 
the  control  acts  in  such  a  manner  as  to  keep  the  state  closer  to  zero  for  successively  larger 
fractions  of  each  “cycle.”  Concomittantly,  we  see  how  the  controls  begin  to  try  to  “invert” 
the  hysteresis,  as  we  just  mentioned. 


Figure  11-3:  Optimal  contn 

•■’I — -w 


,  / ,  ; 

0.05-  ,7\\ 


Figure  11-4:  Comparison  of 
dashed,  q  =  200;  dotted,  q  =  ( 


Figure  11-5:  Comparison  of 
dashed,  q  =  200;  when  ^  =  0, 
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Next,  we  compare  the  solutions  found  above  with  those  computed  by  the  generalized 
value  iteration  and  linear  programming  algorithms  of  §11.1.  In  Figure  11-6  we  plot  the 
optimal  control  again,  but  this  time  we  compare  those  previously  shown  (computed  using 
the  boundary- value  algorithm)  with  those  obtained  using  generalized  value  iteration  arising 
from  the  Bellman  equation  described  in  §11.1.  While  the  surfaces  do  look  similar,  we 
also  plot  several  ‘‘slices”  for  better  comparison.  Up  to  the  discretization  used  in  the  value 

iteration  algorithm,  the  plots  are  seen  to  agree. 

Next,  we  compare  the  optimal  costs  computed  by  linear  programming  with  those  com¬ 
puted  with  value  iteration  for  successively  tighter  convergence  criteria.  See  Figure  11-7. 
Note  that  the  value  iteration  solutions  are  converging  to  those  found  via  linear  program¬ 
ming.  Both  algorithms  used  the  same  discretization  and  the  same  computer  resources  (Mat- 
lab  on  a  Sparc5  after  data  initialization  using  a  C  program).  In  this  case,  the  wall  time 
to  compute  each  cutvg  showti  was  about  equal.  Thus  for  this  problem  linear  programming 
yielded  more  than  a  factor  of  four  speed-up. 


Optimal  Policy  vs.  x  and  q;  Boundary-Value  Problem  Optimal  Policy  vs.  x  and  q:  Value  Iteration 


Figure  11-6:  Comparison  of  policies  computed  by  boundary- value  and  value  iteration.  Slices 
show  details.  Smooth,  boundary- value;  staircase,  value  iteration 


§11.2  Controlling  Example  Systems  173 


Figure  11-7:  Cost-to-go  for  hysteresis  example  versus  state  (s  =  1,-1  superimposed).  Plot 
shows  the  convergence  of  value  iteration  solutions  (dash-dot  and  dotted  lines)  to  that  found 
by  linear  programming  (solid  and  dashed). 


§11.2.2  EXAMPLES  WITH  CONTROLLED  SWITCHING 

We  have  also  considered  two  problems  involving  controlled  switching,  a  satellite  station¬ 
keeping  problem  and  a  transmission  control  problem.  The  second  also  involves  continuous 
controls. 

SATELLITE  CONTROL 

Recall  Example  3.3.  It  is  a  system  with  controlled  switching.  We  add  switching  costs  of 
Con  to  switch  from,  and  Coff  to  switch  to,  i;  =  0.  We  also  penalize  control  and  state  with  a 
running  cost  oi  k  =  cHwH  -F  x?. 

We  have  solved  the  above  using  generalized  policy  iteration  on  Equation  (11.1).  We 
discretize  the  state-space  and  use  the  techniques  described  in  [44]  for  dealing  with  edge 
effects.  The  resulting  solution  confirms  the  banded  structure  of  positive,  zero,  and  negative 
controls,  that  is  typical  for  such  problems.  The  reasoning  is  that  there  are  long  periods  of 
drift  followed  by  short  bursts  of  control  [145].  See  Figure  11-8. 

TRANSMISSION  EXAMPLE 

We  now  consider  the  transmission  system  of  Example  3.4.  We  use  a{y)  =  tanh(j//10)  to 
represent  our  “torque  curve.”  As  a  control  problem  we  consider  minimizing  the  running 
cost 

fe  =  (40-X2)^ 

starting  from  xi  =  X2  —  0.  One  would  expect  that  this  leads  to  an  optimal  acceleration 
strategy  for  reaching  a  velocity  of  40  as  quickly  as  possible.  We  solved  the  problem  using 
both  value  iteration  and  linear  programming  as  outlined  above.  Figure  11-9  shows  the 
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Figure  11-8:  Satellite  Station-Keeping  Policy.  State  in  C,,  denotes  the  policy  is  to  use  v  —  q. 


results.  The  piecewise  shifting  strategies  are  interpreted  as  follows.  The  continuous  control 
u  is  given  by  the  fractional  part,  and  the  gear  shift  position  by  the  integer  part  plus  1. 

The  “analytic  strategy”  shown  results  in  maximum  acceleration  (up  to  the  discretization 
of  velocity  and  control  used).  In  this  case,  it  is  simply  given  (as  a  function  of  X2)  by 

arg  mdixf{x2,u,v)  =  [—a{x2/v)  +  u]/(l  +  v). 

U^V 

The  “calculated  strategy”  was  the  result  of  solving  the  linear  program  described  in  the 
previous  section  using  the  running  cost  above.  Except  for  the  edge  effect,  the  calculated 
and  analytic  shifting  strategies  agree  up  to  the  discretization  used.  This  also  confirms  that 
the  simple  chosen  cost  was  a  good  choice  for  approximating  an  optimal  acceleration  strategy. 
Further,  we  also  plot  four  more  curves,  namely,  the  integral  of  the  controlled  vector  field 

versus  both  time  and  velocity  for  both  the  calculated  and  analytic  solutions.  In  both  cases, 
the  “performance”  of  the  calculated  and  analytic  solutions  are  nearly  identical. 


Figure  11-9:  Optimal  acceleration  strategies  and  performance  for  the  transmission  problem. 
Solid,  calculated;  dashed,  analytic. 


Chapter  12 

Conclusions  and  Future  Work 


This  chapter  summarizes  the  contributions  of  the  thesis  and  gives  a  view  toward  future 
areas  of  research. 


§12.1  CONCLUSIONS 

This  thesis  studied  different  aspects  of  hybrid  systems,  concentrating  in  the  broad  areas  of 
modeling,  analysis,  and  control. 

MODELING 

Review.  In  §3  we  first  identified  the  types  of  discrete  phenomena  that  arise  in  hybrid  sys¬ 
tems.  Then  we  reviewed  models  of  hybrid  systems  firom  the  systems  and  control  literature. 
We  made  some  comparisons  which  enabled  us  to  quickly  prove  simulation  and  modeling 
results  for  such  systems. 

Classification.  §4  presented  our  own  taxonomy  of  hybrid  systems  models.  We  introduced 
four  main  clcisses  of  systems: 

•  general  hybrid  dynamical  systems 

•  hybrid  dynamical  systems,  or  hybrid  systems, 

•  switched  systems 

•  continuous  switched  systems 

Both  autonomous  and  controlled  versions  were  introduced.  Further  classification  was  based 
on  their  structural  properties  and  the  discrete  phenomena  they  exhibit.  We  also  gave  explicit 
instructions  for  computing  the  orbits  and  trajectories  of  general  hybrid  dynamical  systems, 
including  sufiicient  conditions  for  existence  and  uniqueness. 

Unified.  In  §5  we  formulated  our  own  unified  firamework  for  hybrid  systems  modeling 
and  control.  We  explicitly  demonstrated  that  our  unified  model  encompasses  the  identified 
discrete  phenomena  and  the  reviewed  models  of  hybrid  systems.  The  controlled  version  of 
the  proposed  model  contains  discrete  and  continuous  states,  dynamics,  and  controls.  It  is 
useful  in  posing  and  solution  of  hybrid  control  problems,  including  ones  with  disturbances. 
This  was  examined  in  Part  III. 
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ANALYSIS 

Topology.  In  §6  we  discussed  some  of  the  topological  issues  that  arise  when  differential 
equations  and  finite  automata  interact  in  hybrid  systems.  We  concentrated  on  the  maps 
from  a  continuum  to  a  finite  symbol  space — AD  maps — and  back  to  another  continuum 
DA  maps.  We  illuminated  the  general  difficulties  with  the  usual  topologies  in  allowing 
continuous  AD  maps,  constructed  several  topologies  which  bypassed  them,  and  examined 
at  length  one  such  topology  due  to  Nerode  and  Kohn. 

We  showed  that  there  are  inherent  limitations  present  when  one  desires  continuous 
maps  from  continuum  to  continuum  through  a  finite  symbol  space,  viz.,  one  must  equip 
the  continua  with  new  topologies.  We  constructed  a  control  space  topology  allowing  a 
continuous  map  which  “completes  the  loop.” 

We  ended  with  a  different  view  of  hybrid  systems  that  may  broach  these  problems.  As 
an  example,  we  showed  that  the  most  widely  used  fuzzy  logic  control  structure  is  related  to 
this  different  view  and  that  it  indeed  is  a  continuous  map  from  measurements  to  controls. 
We  further  demonstrated  that  these  fuzzy  logic  controllers  are  dense  in  the  set  of  such 
continuous  functions. 

We  also  made  some  connections  with  hybrid  system  trajectories  and  definitions  of  sim¬ 
ulation. 

Complexity.  In  §7  we  explored  the  simulation  and  computational  capabilities  of  hybrid 
systems.  To  accomplish  this,  we  first  defined  notions  of  simulation  of  a  discrete  dynamical 
system  by  a  continuous  dynamical  system.  S-simulation,  or  simulation  via  section,  was 
motivated  by  the  definition  of  global  section  in  dynamical  systems  [129].  Relaxing  this 
to  allow  different  parameterizations  of  time  we  considered  P-simulation  (simulation  via 
points),  which  was  seen  to  be  weak.  To  remedy  this,  we  defined  Tsimulation,  or  simulation 
via  intervals.  Both  S-simulation  and  I-simulation  imply  P-simulation.  S-simulation  and 
I-simulation  are  independent  notions. 

We  then  showed  that  hybrid  systems  models  with  the  ability  to  implement  an  exact  clock 
can  simulate  fairly  general  discrete  dynamical  systems.  Namely,  we  demonstrated  that  such 
systems  can  S-simulate  arbitrary  reversible  discrete  dynamical  systems  defined  on  closed 
subsets  of  R".  These  simulations  require  ODEs  in  R^"  with  the  exact  clock  as  input.  Each 
of  the  reviewed  hybrid  systems  models  can  implement  exact  clocks.  They  require  only  the 
most  benign  hybrid  systems:  two  discrete  states  and  autonomous  switching. 

Later,  we  found  that  one  can  simulate  arbitrary  discrete  dynamical  systems  defined  on 
subsets  of  Z”  without  the  capability  of  implementing  an  exact  clock.  Instead,  one  can  use  an 
approximation  to  an  exact  clock,  implemented  with  a  one-dimensional  Lipschitz  ODE.  The 
result  is  that  we  can  perform  STsimulations  (resp.  I-simulations)  using  continuous  (resp. 
Lipschitz)  ODEs  in  . 

Turning  to  computational  abilities,  we  saw  that  there  are  systems  of  continuous  ODEs 
possessing  the  ability  to  Sl-simulate  arbitrary  pushdown  automata  and  Turing  machines. 
Finite  automata  may  be  Sl-simulated  with  continuous,  Lipschitz  ODEs.  By  Sl-simulating  a 
universal  Turing  machine,  we  concluded  that  there  are  ODEs  in  R^  with  continuous  vector 
fields  possessing  the  power  of  universal  computation. 

Finally,  we  showed  that  hybrid  systems  are  strictly  more  powerful  than  Lipschitz  ODEs 
in  the  types  of  systems  they  can  implement.  For  this,  we  used  a  nontrivial  example:  the 
famous  asynchronous  arbiter  problem.  First,  we  settled  the  problem  in  an  ODE  framework 
by  showing  one  cannot  build  an  arbiter  with  devices  modeled  by  Lipschitz  ODEs  continuous 
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in  inputs  and  outputs.  Then,  we  showed  that  each  of  the  reviewed  models  of  hybrid  systems 
can  implement  arbiters,  even  when  their  continuous  dynamics  are  modeled  by  Lipschitz 
ODEs  continuous  in  inputs  and  outputs.  Again,  such  examples  require  only  autonomous 
switching  and  two  discrete  states. 


Analysis  Tools.  §8  detailed  work  on  switched  and  continuous  switched  systems.  Eax;h 
arise  from  hybrid  systems  by  abstracting  away  the  finite  dynamics.  In  the  first  part  of 
§8,  we  developed  general  tools  for  analyzing  continuous  switching  systems.  For  instance, 
we  proved  an  extension  of  Bendbcson’s  Theorem  to  the  case  of  Lipschitz  continuous  vector 
fields.  This  gives  us  a  tool  for  analyzing  the  existence  of  limit  cycles  of  continuous  switching 
systems.  We  also  proved  a  lemma  dealing  with  the  continuity  of  differential  equations  with 
respect  to  perturbations  that  preserve  a  linear  part.  Colloquially,  this  Linear  Robustness 
Lemma  demonstrates  the  robustness  of  ODEs  with  a  linear  part.  The  lemma  is  useful  in 
easily  deriving  some  of  the  common  robustness  results  of  nonlinear  ODE  theory  (as  given 
in,  for  instance,  [11]).  It  also  becomes  useful  in  studying  singular  perturbations  if  the  fast 
dynamics  are  such  that  they  maintain  the  corresponding  algebraic  equation  to  within  a 
small  deviation.  We  added  some  simple  propositions  that  allowed  us  to  do  this  type  of 
analysis  in  §9. 

In  the  second  part  of  §8,  we  introduced  “multiple  Lyapunov  functions”  as  a  tool  for 
analyzing  Lyapimov  stability  of  switched  systems.  The  idea  here  is  to  impose  conditions 
on  switching  that  guarantee  stability  when  we  have  Lyapunov  functions  for  each  system  fi 
individually.  Also,  iterative  function  systems  were  presented  as  a  tool  for  proving  Lagrange 
stability  and  positive  invariance.  We  also  address  the  case  where  the  finite  index  set  is 
replaced  by  an  arbitrary  compact  set. 


Analyzing  Examples.  In  §9.2  we  presented  an  example  hybrid  control  problem:  the  max 
system.  This  system  was  inspired  from  one  used  in  the  control  of  modern  aircraft.  The 
control  law  uses  a  logical  function  (max)  to  pick  between  one  of  two  stable  controllers: 
one  a  servo  that  tracks  pilot  inputs,  the  second  a  regulator  about  a  fixed  angle  of  attack. 
Typically,  engineers  resort  to  extensive  simulation  of  even  such  simple  systems  because  the 
analysis  is  too  hard  with  their  present  toolbox.  However,  we  analyzed  the  stability  of  this 
hybrid  system  in  the  case  where  the  pilot  input  is  zero  and  the  controllers  are  linear  full- 
state  feedback.  We  showed  that  no  limit  cycles  exist  by  proving  and  applying  an  extension 
of  Bendixson’s  Theorem  to  the  case  of  Lipschitz  continuous  vector  fields;  we  also  gave  a 
Lyapunov  function  that  proved  all  systems  of  this  form  axe  globally  asymptotically  stable. 
Interestingly,  the  Lyapunov  equation  used  a  logical  switching. 

In  §9.3  we  presented  an  analysis  of  a  “continuation”  of  the  max  system.  That  is,  we 
used  a  differential  equation  to  obtain  a  “smooth”  function  instead  of  using  the  output  given 
by  the  max  function  directly.  By  extending  a  result  in  the  theory  of  continuity  of  solutions 
of  ordinary  differential  equations,  we  proved  stability  properties  of  the  continuation  from 
those  of  the  original  max  system. 

The  conclusion  in  this  case  is  that  the  continuation  method  worked  in  reverse,  i.e.,  it  was 
easier  to  prove  stability  of  the  original,  hybrid  system  directly.  Furthermore,  we  concluded 
stability  of  the  continuation  via  that  of  the  original  system.  In  effect,  we  showed  robustness 
of  the  max  system  to  the  considered  class  of  dynamic  continuations. 
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CONTROL 

Theoretical  Results.  In  this  part  of  the  thesis,  we  took  an  optimal  control  approach  to 
hybrid  systems.  In  §10  we  defined  an  optimal  control  problem  in  our  unified  hybrid  control 
framework  and  derived  some  theoretical  results.  The  problem,  and  all  assumptions  used  in 
obtaining  the  remaining  results,  were  expressly  stated.  The  necessity  of  these  assumptions— 
or  ones  like  them— was  demonstrated  with  examples.  The  main  results  were  as  follows:  The 
existence  of  optimal  and  e-optimal  controls  for  the  problem  is  established  in  §10.2.  §10.3 
gave  a  formal  derivation  of  the  associated  generalized  quasi-variational  inequalities. 

Algorithms  and  Examples.  Using  the  GQVIs  as  a  starting  point,  §11.1  concentrated  on 
algorithms  for  solving  hybrid  control  problems  by  solving  the  associated  GQVIs.  Our  unified 
view  led  to  the  concept  of  examining  a  generalized  Bellman  equation.  We  also  drew  explicit 
relations  with  impulse  control  of  piecewise-deterministic  processes.  Four  algorithms  were 
outlined:  boundary-value  algorithm,  generalized  value  iteration,  an  impulse  control-like 
approach,  and  linear  programming. 

Three  illustrative  examples  were  solved.  We  first  considered  a  hysteresis  system  that 
exhibits  autonomous  switching  and  has  a  continuous  control.  Then  we  discussed  a  satel¬ 
lite  station-keeping  problem  involving  controlled  switching.  We  ended  with  a  transmission 
problem  with  continuous  accelerator  input  and  discrete  gear-shift  position.  In  each  case, 
the  optimal  controls  produced  verify  engineering  intuition. 

§12.2  FUTURE  WORK 

We  have  certainly  seen  a  broad  range  of  application  areas  to  hybrid  systems.  We  have  only 
opened  the  door  a  little  wider  to  further  investigation.  This  should  proceed  along  three 
fronts  simultaneously:  theory,  applications,  and  development  of  engineering  tools. 

Modeling.  One  needs  to  explore  the  plethora  of  modeling  choices  available  in  hybrid  sys¬ 
tems.  Since  hybrid  systems  include  dynamical  systems  as  a  subset,  subclasses  which  permit 
efficient  simulation,  analysis,  and  verification  should  be  explored.  We  believe  that  such  a 
program  is  indeed  being  carried  out  by  the  computer  scientists.  Control  theorists  should 
do  the  same  in  their  field  in  examining  the  hybrid  control  of  hybrid  systems. 

Analysis.  First,  it  is  not  hard  to  generalize  our  Multiple  Lyapunov  function  (MLF)  theory 
to  the  case  of  different  equilibria,  which  is  generally  the  case  in  hybrid  systems  (see  §8.7). 
For  example,  tmder  a  Lyapunov-like  switching  rule,  after  all  controllers  have  been  switched 
in  at  level  at,  the  set  is  invariant.  Such  a  generalization  is  useful  in  hybrid 

systems,  where  different  equilibrium  generally  arise.  In  this  case,  the  multiple  Lyapunov 
approach  appears  useful  in  establishing  the  convergence  of  optimization  algorithms  which 
perform  jumps  upon  entering  certain  regions  of  the  state-space  [136] 

Since  sufficient  conditions  for  stability  can  become  guidelines  for  synthesis,  control  design 
arising  from  the  constraints  of  §8  is  a  topic  of  further  research.  Following  our  example  above, 
it  can  lead  to  the  design  of  convergent  algorithms. 

There  are  also  theoretical  issues  to  be  explored.  Some  examples  include  the  stability 
of  systems  with  multiple  equilibrium  points,  the  stability  of  switched  systems,  relations 
between  fixed-point  theory  and  Lyapunov  stability,  and  the  stability  and  dynamics  of  or¬ 
dinary  differential  equations  driven  by  Markov  chains  whose  transition  probabilities  are  a 
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function  of  the  continuous  state.  The  latter  may  provide  a  link  to  the  large  literature  on 
jump  systems  (see  §2.2.2). 

Another  important  topic  of  further  research  is  to  incorporate  developed  analysis  tools 
into  software  engineering  tools.  This  will  allow  application  of  these  tools  to  complicated 
examples  in  a  timely  manner. 

Control.  Specific  open  theoretical  issues  were  discussed  in  §10.4.  Another  has  to  do  with 
the  robustness  of  our  hybrid  controls  with  respect  to  state.  Here,  our  transversality  assump¬ 
tions  should  combine  with  Tavernini’s  result  on  continuity  with  respect  initial  condition  to 
yield  continuity  of  control  laws  on  an  open  dense  set. 

An  important  area  of  current  research  is  to  develop  good  computational  schemes  to 
compute  near-optimal  controls  in  realistic  cases.  Analysis  of  rates  of  convergence  of  dis¬ 
cretized  algorithms  should  be  explored.  Later,  the  development  of  software  tools  to  design 
such  controllers  automatically  will  become  an  important  area  of  research. 

On  to  Design.  Finally,  from  modeling,  through  analysis  and  control,  we  come  to  design 
of  complex,  hybrid  systems.  Here,  some  of  the  interaction  between  levels  is  under  our 
jurisdiction.  What  would  we  do  with  such  freedom,  coupled  with  our  new-found  analysis 
and  control  techniques?  For  example,  we  might  design  a  flexible  manufacturing  system  that 
not  only  allows  quick  changes  between  different  product  lines,  but  allows  manufacturing  of 
new  products  on  the  line  with  relative  ease. 

Consider  the  so-called  reflex  controllers  of  [116,  148],  which  constitute  a  dynamically 
consistent  interface  between  low-level  servo  control  and  higher-level  planning  algorithms 
that  ensures  obstacle  avoidance.  Thus  as  a  step  in  the  direction  of  hierarchical,  hybrid 
design,  the  reflex  control  concept  is  an  example  of  how  to  incorporate  a  new  control  module 
to  allow  rapid,  dynamically  transparent  design  of  higher-level  programs.  Further,  there 
are  some  structures  for  the  control  programs  used  in  research  aircraft  [79]  that  may  lend 
themselves  to  such  an  approach.  In  each  case,  these  designs  incorporate  structures  which 
allow  engineers  to  separate  the  continuous  and  logical  worlds. 

These  controllers  provide  the  inspiration,  our  analysis  and  control  results  the  foundation, 
and  our  steps  toward  efficient  algorithms  the  impetus,  for  setting  a  course  toward  design 
of  hybrid  systems.  Ultimately,  it  is  hoped  they  will  lead  to  truly  intelligent  engineering 
systems. 
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Appendix  A 

Topology  Review 


The  following  is  only  a  quick  review  (and  we  do  assume  some  basic  point  set  topology,  e.g, 
[126,  Ch.  2]).  For  more  details  consult  [62,  75,  113]. 

A  topological  space  consists  of  a  set  X  and  a  topology  on  X.  A  topology  on  X  is 
a  set  T  of  designated  subsets  of  X,  called  open  sets,  such  that 

•  0,  X  are  in  T, 

•  T  is  closed  under  arbitrary  unions, 

•  T  is  closed  under  finite  intersections. 

EXAMPLE.  77  =  {0,  A}  and  77)  =  2-^  are  each  topologies  on  X,  known  as  the  indiscrete 
and  discrete  topologies,  respectively. 

If  y  is  a  subset  of  topological  space  (X,  T),  the  collection 

TY  =  {YnU\U€T} 

is  a  topology  on  Y  called  the  subspace  topology.  With  this  topology,  Y  is  called  a 
subspace  of  X,  and  17  C  X  is  open  in  Y  (or  open  relative  to  Y)  if  it  belongs  to  7y. 

A  set  C  C  X  is  called  closed  if  X\C  is  open.  A  set  can  be  open,  closed,  both,  or 
neither.  Suppose  T  and  'T'  are  two  topologies  on  a  given  set  X.  Yi'T'  (Z'T-,  we  say  that  T' 
is  finer  than  T,  and  that  T  is  coarser  than  T'. 

If  X  is  a  set,  a  basis  for  a  topology  on  X  is  a  collection  B  of  subsets  of  X  (called  basis 
elements)  such  that 

1.  Each  re  €  X  is  contained  in  at  least  one  element  of  B. 

2.  If  X  belongs  to  the  intersection  of  two  basis  elements  Bi  2md  B2,  then  there  exists  a 
basis  element  S3  C  Bi  n  B2  containing  x. 

A  subbasis  S  for  a  topology  on  X  is  a  collection  of  subsets  of  X  whose  union  equals  X. 
The  topology  generated  by  basis  B  (subbasis  S)  is  the  collection  of  all  unions  of  (finite 
intersections  of)  elements  of  B  (S). 

EXAMPLE.  For  any  set  X,  the  collection  of  all  one-point  sets  of  X  is  a  basis  for  the  discrete 
topology  on  X.  The  open  rays  of  an  ordered  set  X,  the  collection 

{x  1  X  >  a},  {x  1  X  <  a},  a  €  X, 

are  a  subbasis  for  the  order  topology  on  X. 
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The  function  f  :  X  Y  is  continuous  if  for  all  V  open  inY,  U  =  f~^{V)  is  open  in 
X‘,  it  is  surjective  if  its  image  is  Y,  injective  if  it  is  one-to-one,  and  bijective  if  both  of 
these  hold;  it  is  a  homeomorphism  if  it  is  a  continuous  bijection  with  continuous  inverse. 

A  topological  space  X  is  connected  if  it  cannot  be  written  as  X  =  AuB,  where  A  and 
B  are  open,  nonempty,  and  disjoint.  Two  useful  results  are  the  following. 

•  A  is  connected  if  and  only  if  X  and  0  eire  the  only  subsets  of  X  which  are  both  open 
and  closed. 

•  The  continuous  image  of  a  connected  space  is  connected. 

The  following  classification  of  topological  spaces  is  common: 

•  Tq.  Given  two  distinct  points  in  a  topological  space  A,  at  least  one  of  them  is  contained 
in  an  open  set  not  containing  the  other. 

•  Ti.  Given  two  distinct  points  in  a  topological  space  X,  each  of  them  is  contained  in 
an  open  set  not  containing  the  other. 

•  Tj  or  Hausdorff.  Given  two  distinct  points  in  a  topological  space  X,  there  are 
disjoint  open  sets,  each  containing  just  one  of  the  two  points. 

The  following  hold  for  spaces  in  which  one  point  sets  are  closed. 

•  Regular.  Given  a  point  and  a  closed  set  disjoint  firom  it,  there  exist  disjoint  open 
sets  containing  each  of  them. 

•  Normal.  Given  two  disjoint  closed  sets,  there  exist  disjoint  open  sets  containing  each 
of  them. 


Figure  A-1:  Visualization  of  separation  axioms. 

A  space  Y  is  said  to  have  the  universal  extension  property  if  for  each  triple  (X,  A,  /), 
where  X  is  a  normal  space,  A  C  X  is  closed,  and  /  :  A  V  is  continuous,  there  exists 
an  extension  of  /  to  a  continuous  map  of  X  into  Y.  For  arbitrary  index  set  J,  R  has  the 
universal  extension  property  [113,  p.  216]. 
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Common  Notation 


x\u,  x-u 


u 

dU 

1^1 

A^B 

fit-) 

C{X,Y) 

v’^ 

l|2:||,  ||a:||2,  \\x\\co 

V/ 


reals,  nonnegative  reals 

integers,  nonnegative  integers 

complement  of  17  in  X 

closure  of  U 

interior  of  U 

boundary  of  U 

cardinality  of  set  A 

A,  B  are  sets  with  |A|  =  |B| 

right-,  left-hand  limits  of  /  at  f 

continuous  functions  from  X  to  y 

transpose  of  v 

arbitrary.  Euclidean,  and  infinity  norm  of  x 
divergence  of  / 


Special  Notation 
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[xj 

W 

w 

[t]p 

9 

[condition] 

!  [condition] 
?  [condition] 
[X,S,f] 
G 
T 
B 
S 


{1,2,. ..,iV} 

greatest  integer  less  than  or  equal  to  x 
least  integer  greater  than  x 

time  less  than  or  equal  to  t  at  which  the  last  jump  occurred 
time  at  which  the  variable  p  last  jumped 
successor  of  q{t) 
predecessor  of  q{t) 

transition  enabled  (event  is  triggered)  if  condition  is  true 

transition  must  be  taken  (event  must  be  accepted)  if  condition  is  true 

transition  may  be  taken  (event  may  be  accepted)  if  condition  is  true 

dynamical/transition  system  /  defined  on  X  over  the  semigroup  S 

continuous  extension  of  G 

topology 

basis 

subbasis 
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Hybrid  Systems  Models 


State  Spaces 

S  hybrid  state  space 
X  continuous  state  space 
Q  discrete  state  space 

Designated  Subsets 

A  autonomous  jump  set 
C  controlled  jump  set 
D  jump  destination  set 
Afg,p  transition  manifolds  [q,  p  6  Q) 

States 

s  hybrid  state 
X  continuous  state 
q  discrete  state 

Dynamics 

(j)g  extended  transition  functions 
fg  transition  functions 
G  jump  transition  maps 

J  impulse  transition  map  (continuous  component  of  G) 
v  switching  transition  map  (discrete  component  of  G) 

Transition  Delays 

Aq  autonomous  jump  delay  map 
Ti  pre-jump  times 
Fj  post-jump  times 

(Tj  pre-jump  times  for  autonomous  jumps 


Controls 

u  continuous  control 

V  discrete  control  (exercised  upon  autonomous  jumps) 
F  controlled  jump  transition  map  (set- valued) 

Ac  controlled  jump  delay  map 

Q  intervention  (pre-jump)  times  for  controlled  jumps 
post-jump  times  for  controlled  jumps 

Costs 

k  running  cost 
Ca  autonomous  jump  cost 
Cc  controlled  jump  cost 
V (•)  value  function 


Index 


AD  map  89 
Aggregation  24 
Alphabet  42 

Anchored  switching  sequence  124 
Arbiter  110 
ASL  63 
ASLAUT  67 
Automatization  25 
Automaton 
digital  44 
differential  58 
finite  42 
hybrid  48 
push-down  43 
symbolic  44 
Autonomous 
-impulse  72 
impulses  53 
jump  53 
jump  set  27 
switching  52 

Basis  181 

elements  181 
sub-  181 

topology  generated  by  181 
BB  64 
BD  65 
BDV  65 

Bellman-Gronwall  lemma  131 
Bendixson’s  theorem  136 
.  extension  120 
BGM  60 
Bijective  182 

C-  71 

see  §  4.4 

Candidate  Lyapimov  function  125 
Canonical  max  system  141 
continuation  144 


Cantor  encodings  104 
Cemetery  states  42 
Clock 

exact  105 
inexact  107 
Closed  181 
Coarser  181 
Connected  182 
simply  136 
Constituent  27 
Contains  67 
Continuation  24 
Continuous  182 

from  the  right  159 
ODEs  42 
Continuous-time 

transition  system  41 
Contractive  fimction  128 
Control  156 
Controlled 

-impulse  72 
impulses  54 
jump  54 

controlled  jump  destination  maps 
jump  set  29 
dynamical  system  41 
switching  53 
vector  field  42 

D-  71 
see  §4.4 
Decision  156 

DEDS  discrete  event  dynamic  system 

Destination  sets,  jump  27 

Destinations  156 

Deterministic  72 

Discrete-time  72 

Digital  automaton  44 

DiflFerential 

automaton  58 
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equation  ordinary  differential  equation 
IFS  130 
inclusion  48 
DIFS  differential  IFS 
Discount  factor  155 
Discrete-event  dynamical  system  see  [74] 
Discrete-time 

transition  system  41 
Discrete  topology  181 
Divergence  120 
Dynamical  system  40 
controlled  41 
equivalent  41 
generator  of  40 
hybrid  hybrid  dynamical  system 
in  (a  space)  40 
with  input  41 
with  marked  states  41 
with  output  41 
with  timing  42 
Dynamically-uniform  72 

Edge-output  41 
Empty  string  42 
Equilibrium  point  41 
Equivalent 

dynamical  systems  41 
Essential  parts  90 
Even  sequence  125 
Exact  clock  105 
Extended  transition  map  40 

FA  finite  automaton 
Final  states  42 
Finer  181 

Finite  automaton  42 
inputless  44 
transition  function  43 
Finite  gain  108 
Fixed  point  41 
Formal  union  155 
From  the  right  159 
continuous  159 

General  hybrid  dynamical  system  27 
Generalized  quasi-variational  inequalities 
(GQVIs)  160 

GHDS  general  hybrid  dynamical  system 
controlled  29 


Global  section  100 

Halting  42 
Hausdorff  182 
HDS  hybrid  dynamical  system 
Homeomorphism  182 
Homomorphic  41 
Hybrid  automaton  48 
Hybrid  dynamical  system  28 
autonomous  72 
autonomous-impulse  72 
autonomous-switching  72 
controlled-impulse  72 
controlled-switching  72 
controlled  72 
general  27 
general  controlled  29 
time-invariant  72 
time-varying  72 
trajectory  of  74 
uncontrolled  72 
Hybrid  system  28 

Identity  property  40 
IFS  iterated  function  systems 
Implements  67 
Impulse 

autonomous  53 
control  46 
controlled  54 
cost  155 

effect,  systems  with  46 
Impulse  effect,  systems  with  systems  with 
impulse  effect 
Index  states  27 
Indiscrete  topology  181 
Inexact  clock  107 
Initial  states  42 
Injective  182 
Inputless 
FA  44 
PDA  44 

Interval  completion  125 
Invariant  41 
I-simulation  101 
Isomorphic 

dynamical  systems  41 
sets  39 
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Iterated  function  system  128 

Join  irreducibles  89 
Jump 

autonomous  53 
controlled  54 
transition  maps  27 
Jump  sets 

autonomous  27 
controlled  29 
delay  map  70 
destination  27 
Jump  systems  45 

Linear  robustness  lemma  121 
Lipschitz  42 
constant  42 
continuous  42 
ODEs  42 
Lyapunov-like  125 

Map  39 
Max  system  140 
canonical  144 
continuation  144 
Mealy  machines  41 
MLF  multiple  Lyapunov  functions 
Monoid  40 
Moore  machines  41 
Multiple  Lyapunov  functions  123 

NKAUT  67 
NKSD  61 
Nonautonomous  72 
Non-degeneracy  108 
Nondeterministic  72 
Normal  182 

ODEs  ordinary  differential  equations 
Open  181 
in  181 

relative  to  181 
sets  181 
rays  181 
Orbit  41 

positive  73 
Order  topology  181 
Ordinary  differential  equations  42 
autonomous  42 


continuous  42 

inputs  42 

Lipschitz  42 

outputs  42, 

time-invariant  42 

with  inputs  and  outputs  42 

PDA  push- down  automaton 
PDF  piecewise-deterministic  process 
Petri  nets  see  [99] 

Piecewise-constant  derivative  systems  116 
Piecewise-deterministic  process  47 
Plant  42 
Pop  43 
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